Wan2.1 T2V 1.3B

提供商Wan-AI
分类text-to-video
许可证apache-2.0
下载量8.1M
星标184

简介

Wan2.1 T2V 1.3B 是一款轻量级且高效的文本生成视频模型。相比于动辄数十亿参数的大模型，它在保持较高生成质量的同时，极大地降低了显存门槛，使得开发者在消费级显卡上也能流畅地进行视频创作和快速迭代。该模型擅长处理短视频片段，能够较好地理解指令并将其转化为动态画面，非常适合作为 AI 视频工作流中的快速原型工具，或者集成到对推理速度要求较高的实时应用场景中。

核心亮点

  • 轻量化架构，消费级显卡即可流畅运行
  • 文本理解力强，快速生成高质量动态视频
  • Apache-2.0 协议，对商业化应用极其友好
  • 适合快速原型开发与低延迟视频生成场景

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B")
tokenizer = AutoTokenizer.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download Wan-AI/Wan2.1-T2V-1.3B

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download Wan-AI/Wan2.1-T2V-1.3B config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('Wan-AI/Wan2.1-T2V-1.3B')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('Wan-AI/Wan2.1-T2V-1.3B')
tokenizer = AutoTokenizer.from_pretrained('Wan-AI/Wan2.1-T2V-1.3B')

模型下载

我们推荐使用命令行或者 ModelScope SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 ModelScope：

操作指引 
pip install modelscope

命令行下载

下载完整模型库

下载完整模型库 
modelscope download --model Wan-AI/Wan2.1-T2V-1.3B

下载单个文件到指定本地文件夹（以下载 README.md 到当前路径下 dir 目录为例）

下载单个文件到指定本地文件夹（以下载 README.md 到当前路径下 dir 目录为例） 
modelscope download --model Wan-AI/Wan2.1-T2V-1.3B README.md --local_dir ./dir

更多更丰富的命令行下载选项，可参见具体文档

SDK 下载

SDK 下载 
# 模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('Wan-AI/Wan2.1-T2V-1.3B')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://www.modelscope.cn/Wan-AI/Wan2.1-T2V-1.3B.git

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://www.modelscope.cn/Wan-AI/Wan2.1-T2V-1.3B.git

ModelScope 模型页直接下载模型文件；无需将模型文件放在本站服务器。

Notebook 快速开发

下载并安装 ModelScope library

下载并安装 ModelScope library 
pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html

模型加载和推理

模型加载和推理 
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

p = pipeline('text-generation', 'Wan-AI/Wan2.1-T2V-1.3B')

完整文档

来源: HuggingFace

---
license: apache-2.0
language:

  • en

  • zh

pipeline_tag: text-to-video
library_name: diffusers
tags:
  • video

  • video-generation

---

Wan2.1

<p align="center">
<img src="assets/logo.png" width="400"/>
<p>

<p align="center">
💜 <a href=""><b>Wan</b></a> &nbsp&nbsp ｜ &nbsp&nbsp 🖥️ <a href="https://github.com/Wan-Video/Wan2.1">GitHub</a> &nbsp&nbsp | &nbsp&nbsp🤗 <a href="https://huggingface.co/Wan-AI/">Hugging Face</a>&nbsp&nbsp | &nbsp&nbsp🤖 <a href="https://modelscope.cn/organization/Wan-AI">ModelScope</a>&nbsp&nbsp | &nbsp&nbsp 📑 <a href="">Paper (Coming soon)</a> &nbsp&nbsp | &nbsp&nbsp 📑 <a href="https://wanxai.com">Blog</a> &nbsp&nbsp | &nbsp&nbsp💬 <a href="https://gw.alicdn.com/imgextra/i2/O1CN01tqjWFi1ByuyehkTSB_!!6000000000015-0-tps-611-1279.jpg">WeChat Group</a>&nbsp&nbsp | &nbsp&nbsp 📖 <a href="https://discord.gg/p5XbdQV7">Discord</a>&nbsp&nbsp
<br>

-----

Wan: Open and Advanced Large-Scale Video Generative Models <be>

In this repository, we present Wan2.1, a comprehensive and open suite of video foundation models that pushes the boundaries of video generation. Wan2.1 offers these key features:

  • 👍 SOTA Performance: Wan2.1 consistently outperforms existing open-source models and state-of-the-art commercial solutions across multiple benchmarks.

  • 👍 Supports Consumer-grade GPUs: The T2V-1.3B model requires only 8.19 GB VRAM, making it compatible with almost all consumer-grade GPUs. It can generate a 5-second 480P video on an RTX 4090 in about 4 minutes (without optimization techniques like quantization). Its performance is even comparable to some closed-source models.

  • 👍 Multiple Tasks: Wan2.1 excels in Text-to-Video, Image-to-Video, Video Editing, Text-to-Image, and Video-to-Audio, advancing the field of video generation.

  • 👍 Visual Text Generation: Wan2.1 is the first video model capable of generating both Chinese and English text, featuring robust text generation that enhances its practical applications.

  • 👍 Powerful Video VAE: Wan-VAE delivers exceptional efficiency and performance, encoding and decoding 1080P videos of any length while preserving temporal information, making it an ideal foundation for video and image generation.

This repository hosts our T2V-1.3B model, a versatile solution for video generation that is compatible with nearly all consumer-grade GPUs. In this way, we hope that Wan2.1 can serve as an easy-to-use tool for more creative teams in video creation, providing a high-quality foundational model for academic teams with limited computing resources. This will facilitate both the rapid development of the video creation community and the swift advancement of video technology.

Video Demos

<div align="center">
<video width="80%" controls>
<source src="https://cloud.video.taobao.com/vod/Jth64Y7wNoPcJki_Bo1ZJTDBvNjsgjlVKsNs05Fqfps.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
</div>

🔥 Latest News!!

  • Feb 25, 2025: 👋 We've released the inference code and weights of Wan2.1.

📑 Todo List

  • Wan2.1 Text-to-Video
- [x] Multi-GPU Inference code of the 14B and 1.3B models - [x] Checkpoints of the 14B and 1.3B models - [x] Gradio demo - [ ] Diffusers integration - [ ] ComfyUI integration
  • Wan2.1 Image-to-Video
- [x] Multi-GPU Inference code of the 14B model - [x] Checkpoints of the 14B model - [x] Gradio demo - [ ] Diffusers integration - [ ] ComfyUI integration

Quickstart

#### Installation
Clone the repo:

code
git clone https://github.com/Wan-Video/Wan2.1.git
cd Wan2.1

Install dependencies:

code
# Ensure torch >= 2.4.0
pip install -r requirements.txt

#### Model Download

| Models | Download Link | Notes |
| --------------|-------------------------------------------------------------------------------|-------------------------------|
| T2V-14B | 🤗 Huggingface 🤖 ModelScope | Supports both 480P and 720P
| I2V-14B-720P | 🤗 Huggingface 🤖 ModelScope | Supports 720P
| I2V-14B-480P | 🤗 Huggingface 🤖 ModelScope | Supports 480P
| T2V-1.3B | 🤗 Huggingface 🤖 ModelScope | Supports 480P

> 💡Note: The 1.3B model is capable of generating videos at 720P resolution. However, due to limited training at this resolution, the results are generally less stable compared to 480P. For optimal performance, we recommend using 480P resolution.

Download models using 🤗 huggingface-cli:

code
pip install "huggingface_hub[cli]"
huggingface-cli download Wan-AI/Wan2.1-T2V-1.3B --local-dir ./Wan2.1-T2V-1.3B

Download models using 🤖 modelscope-cli:

code
pip install modelscope
modelscope download Wan-AI/Wan2.1-T2V-1.3B --local_dir ./Wan2.1-T2V-1.3B

#### Run Text-to-Video Generation

This repository supports two Text-to-Video models (1.3B and 14B) and two resolutions (480P and 720P). The parameters and configurations for these models are as follows:

<table>
<thead>
<tr>
<th rowspan="2">Task</th>
<th colspan="2">Resolution</th>
<th rowspan="2">Model</th>
</tr>
<tr>
<th>480P</th>
<th>720P</th>
</tr>
</thead>
<tbody>
<tr>
<td>t2v-14B</td>
<td style="color: green;">✔️</td>
<td style="color: green;">✔️</td>
<td>Wan2.1-T2V-14B</td>
</tr>
<tr>
<td>t2v-1.3B</td>
<td style="color: green;">✔️</td>
<td style="color: red;">❌</td>
<td>Wan2.1-T2V-1.3B</td>
</tr>
</tbody>
</table>

##### (1) Without Prompt Extention

To facilitate implementation, we will start with a basic version of the inference process that skips the prompt extension step.

  • Single-GPU inference
code
python generate.py  --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --sample_shift 8 --sample_guide_scale 6 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."

If you encounter OOM (Out-of-Memory) issues, you can use the --offload_model True and --t5_cpu options to reduce GPU memory usage. For example, on an RTX 4090 GPU:

code
python generate.py  --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --offload_model True --t5_cpu --sample_shift 8 --sample_guide_scale 6 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."

> 💡Note: If you are using the T2V-1.3B model, we recommend setting the parameter --sample_guide_scale 6. The --sample_shift parameter can be adjusted within the range of 8 to 12 based on the performance.

  • Multi-GPU inference using FSDP + xDiT USP
code
pip install "xfuser>=0.4.1"
torchrun --nproc_per_node=8 generate.py --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --dit_fsdp --t5_fsdp --ulysses_size 8 --sample_shift 8 --sample_guide_scale 6 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."

##### (2) Using Prompt Extention

Extending the prompts can effectively enrich the details in the generated videos, further enhancing the video quality. Therefore, we recommend enabling prompt extension. We provide the following two methods for prompt extension:

  • Use the D
查看官方来源