Wan2.1 T2V 1.3B Diffusers

提供商Wan-AI
分类text-to-video
许可证apache-2.0
下载量12.6K
星标14

简介

Wan2.1 T2V 1.3B 是一个轻量级的文本生成视频模型，现已集成至 Diffusers 库中，极大降低了开发者的部署门槛。相比于动辄数十 B 参数的视频大模型，1.3B 版本在保持不错画质的同时，大幅降低了显存占用，使得在消费级 GPU 上进行快速迭代和微调成为可能。它非常适合需要快速出片、对实时性有要求，或者希望在本地搭建简单视频生成管线的开发者。如果你熟悉 Stable Diffusion 的 Diffusers 流程，上手这个模型几乎没有学习成本。

核心亮点

  • 轻量级参数量，消费级显卡即可流畅运行
  • 集成 Diffusers 库，代码调用简单且灵活
  • 支持文本直接生成视频，出片速度快
  • Apache-2.0 协议，商业化部署非常友好

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")
tokenizer = AutoTokenizer.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download Wan-AI/Wan2.1-T2V-1.3B-Diffusers

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download Wan-AI/Wan2.1-T2V-1.3B-Diffusers config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('Wan-AI/Wan2.1-T2V-1.3B-Diffusers')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B-Diffusers

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B-Diffusers

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('Wan-AI/Wan2.1-T2V-1.3B-Diffusers')
tokenizer = AutoTokenizer.from_pretrained('Wan-AI/Wan2.1-T2V-1.3B-Diffusers')

模型下载

我们推荐使用命令行或者 ModelScope SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 ModelScope：

操作指引 
pip install modelscope

命令行下载

下载完整模型库

下载完整模型库 
modelscope download --model Wan-AI/Wan2.1-T2V-1.3B-Diffusers

下载单个文件到指定本地文件夹（以下载 README.md 到当前路径下 dir 目录为例）

下载单个文件到指定本地文件夹（以下载 README.md 到当前路径下 dir 目录为例） 
modelscope download --model Wan-AI/Wan2.1-T2V-1.3B-Diffusers README.md --local_dir ./dir

更多更丰富的命令行下载选项，可参见具体文档

SDK 下载

SDK 下载 
# 模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('Wan-AI/Wan2.1-T2V-1.3B-Diffusers')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://www.modelscope.cn/Wan-AI/Wan2.1-T2V-1.3B-Diffusers.git

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://www.modelscope.cn/Wan-AI/Wan2.1-T2V-1.3B-Diffusers.git

ModelScope 模型页直接下载模型文件；无需将模型文件放在本站服务器。

Notebook 快速开发

下载并安装 ModelScope library

下载并安装 ModelScope library 
pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html

模型加载和推理

模型加载和推理 
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

p = pipeline('text-generation', 'Wan-AI/Wan2.1-T2V-1.3B-Diffusers')

完整文档

来源: HuggingFace

---
license: apache-2.0
language:

  • en

  • zh

pipeline_tag: text-to-video
library_name: diffusers
tags:
  • video

  • video-generation

---

Wan2.1

<p align="center">
<img src="assets/logo.png" width="400"/>
<p>

<p align="center">
💜 <a href=""><b>Wan</b></a> &nbsp&nbsp ｜ &nbsp&nbsp 🖥️ <a href="https://github.com/Wan-Video/Wan2.1">GitHub</a> &nbsp&nbsp | &nbsp&nbsp🤗 <a href="https://huggingface.co/Wan-AI/">Hugging Face</a>&nbsp&nbsp | &nbsp&nbsp🤖 <a href="https://modelscope.cn/organization/Wan-AI">ModelScope</a>&nbsp&nbsp | &nbsp&nbsp 📑 <a href="">Paper (Coming soon)</a> &nbsp&nbsp | &nbsp&nbsp 📑 <a href="https://wanxai.com">Blog</a> &nbsp&nbsp | &nbsp&nbsp💬 <a href="https://gw.alicdn.com/imgextra/i2/O1CN01tqjWFi1ByuyehkTSB_!!6000000000015-0-tps-611-1279.jpg">WeChat Group</a>&nbsp&nbsp | &nbsp&nbsp 📖 <a href="https://discord.gg/p5XbdQV7">Discord</a>&nbsp&nbsp
<br>

-----

Wan: Open and Advanced Large-Scale Video Generative Models <be>

In this repository, we present Wan2.1, a comprehensive and open suite of video foundation models that pushes the boundaries of video generation. Wan2.1 offers these key features:

  • 👍 SOTA Performance: Wan2.1 consistently outperforms existing open-source models and state-of-the-art commercial solutions across multiple benchmarks.

  • 👍 Supports Consumer-grade GPUs: The T2V-1.3B model requires only 8.19 GB VRAM, making it compatible with almost all consumer-grade GPUs. It can generate a 5-second 480P video on an RTX 4090 in about 4 minutes (without optimization techniques like quantization). Its performance is even comparable to some closed-source models.

  • 👍 Multiple Tasks: Wan2.1 excels in Text-to-Video, Image-to-Video, Video Editing, Text-to-Image, and Video-to-Audio, advancing the field of video generation.

  • 👍 Visual Text Generation: Wan2.1 is the first video model capable of generating both Chinese and English text, featuring robust text generation that enhances its practical applications.

  • 👍 Powerful Video VAE: Wan-VAE delivers exceptional efficiency and performance, encoding and decoding 1080P videos of any length while preserving temporal information, making it an ideal foundation for video and image generation.

This repository hosts our T2V-1.3B model, a versatile solution for video generation that is compatible with nearly all consumer-grade GPUs. In this way, we hope that Wan2.1 can serve as an easy-to-use tool for more creative teams in video creation, providing a high-quality foundational model for academic teams with limited computing resources. This will facilitate both the rapid development of the video creation community and the swift advancement of video technology.

Video Demos

<div align="center">
<video width="80%" controls>
<source src="https://cloud.video.taobao.com/vod/Jth64Y7wNoPcJki_Bo1ZJTDBvNjsgjlVKsNs05Fqfps.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
</div>

🔥 Latest News!!

  • Feb 25, 2025: 👋 We've released the inference code and weights of Wan2.1.

📑 Todo List

  • Wan2.1 Text-to-Video
- [x] Multi-GPU Inference code of the 14B and 1.3B models - [x] Checkpoints of the 14B and 1.3B models - [x] Gradio demo - [x] Diffusers integration - [ ] ComfyUI integration
  • Wan2.1 Image-to-Video
- [x] Multi-GPU Inference code of the 14B model - [x] Checkpoints of the 14B model - [x] Gradio demo - [x] Diffusers integration - [ ] ComfyUI integration

Quickstart

#### Installation
Clone the repo:

code
git clone https://github.com/Wan-Video/Wan2.1.git
cd Wan2.1

Install dependencies:

code
# Ensure torch >= 2.4.0
pip install -r requirements.txt

#### Model Download

| Models | Download Link | Notes |
| --------------|-------------------------------------------------------------------------------|-------------------------------|
| T2V-14B | 🤗 Huggingface 🤖 ModelScope | Supports both 480P and 720P
| I2V-14B-720P | 🤗 Huggingface 🤖 ModelScope | Supports 720P
| I2V-14B-480P | 🤗 Huggingface 🤖 ModelScope | Supports 480P
| T2V-1.3B | 🤗 Huggingface 🤖 ModelScope | Supports 480P

> 💡Note: The 1.3B model is capable of generating videos at 720P resolution. However, due to limited training at this resolution, the results are generally less stable compared to 480P. For optimal performance, we recommend using 480P resolution.

Download models using 🤗 huggingface-cli:

code
pip install "huggingface_hub[cli]"
huggingface-cli download Wan-AI/Wan2.1-T2V-1.3B-Diffusers --local-dir ./Wan2.1-T2V-1.3B-Diffusers

Download models using 🤖 modelscope-cli:

code
pip install modelscope
modelscope download Wan-AI/Wan2.1-T2V-1.3B-Diffusers --local_dir ./Wan2.1-T2V-1.3B-Diffusers

#### Run Text-to-Video Generation

This repository supports two Text-to-Video models (1.3B and 14B) and two resolutions (480P and 720P). The parameters and configurations for these models are as follows:

<table>
<thead>
<tr>
<th rowspan="2">Task</th>
<th colspan="2">Resolution</th>
<th rowspan="2">Model</th>
</tr>
<tr>
<th>480P</th>
<th>720P</th>
</tr>
</thead>
<tbody>
<tr>
<td>t2v-14B</td>
<td style="color: green;">✔️</td>
<td style="color: green;">✔️</td>
<td>Wan2.1-T2V-14B</td>
</tr>
<tr>
<td>t2v-1.3B</td>
<td style="color: green;">✔️</td>
<td style="color: red;">❌</td>
<td>Wan2.1-T2V-1.3B</td>
</tr>
</tbody>
</table>

##### (1) Without Prompt Extention

To facilitate implementation, we will start with a basic version of the inference process that skips the prompt extension step.

  • Single-GPU inference
code
python generate.py  --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --sample_shift 8 --sample_guide_scale 6 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."

If you encounter OOM (Out-of-Memory) issues, you can use the --offload_model True and --t5_cpu options to reduce GPU memory usage. For example, on an RTX 4090 GPU:

code
python generate.py  --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --offload_model True --t5_cpu --sample_shift 8 --sample_guide_scale 6 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."

> 💡Note: If you are using the T2V-1.3B model, we recommend setting the parameter --sample_guide_scale 6. The --sample_shift parameter can be adjusted within the range of 8 to 12 based on the performance.

  • Multi-GPU inference using FSDP + xDiT USP
code
pip install "xfuser>=0.4.1"
torchrun --nproc_per_node=8 generate.py --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --dit_fsdp --t5_fsdp --ulysses_size 8 --sample_shift 8 --sample_guide_scale 6 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."

Wan can also be run directly using 🤗 Diffusers!

```python
import torch
from diffusers import AutoencoderKLWan, WanPipeline
from diffusers.utils import export_to_video

Available models: Wan-AI/Wan2.1-T2V-14B-Diffusers, Wan-AI/Wan2.1

查看官方来源