使用通义万相Wan2.1进行视频生成

最近通义万相开源了其视频生成模型。模型有两个版本，一个是1.3B的，一个是14B的。对于1.3B的版本，模型大小约17G，14B的版本模型大小约70G。由于显存限制，下面测试了1.3B版本的文生视频。

总体来说，1.3B的版本，实测的视频生成内容，还是有些粗糙，也许1.3B的版本，只是用于让大家熟悉和学习的。小伙伴们如果有条件的，可以测试一下14B的视频生成结果。

下面是我测试过程的一些记录，供大家参考。

源代码准备

git clone https://github.com/Wan-Video/Wan2.1.git

(base) ubuntu@ubuntu-server:~/code$ git clone https://github.com/Wan-Video/Wan2.1.git
Cloning into 'Wan2.1'...
remote: Enumerating objects: 89, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 89 (delta 13), reused 7 (delta 5), pack-reused 70 (from 1)
Receiving objects: 100% (89/89), 5.74 MiB | 1.68 MiB/s, done.
Resolving deltas: 100% (29/29), done.
(base) ubuntu@ubuntu-server:~/code$ 

运行环境准备

创建Python虚拟环境并激活

conda create -n wan python=3.10
conda activate wan

安装依赖包

pip install -r requirements.txt

安装后的依赖包信息

(wan) ubuntu@ubuntu-server:~$ pip list
Package                  Version
------------------------ -----------
accelerate               1.4.0
aiofiles                 23.2.1
aiohappyeyeballs         2.4.6
aiohttp                  3.11.13
aiosignal                1.3.2
annotated-types          0.7.0
anyio                    4.8.0
async-timeout            5.0.1
attrs                    25.1.0
certifi                  2025.1.31
charset-normalizer       3.4.1
click                    8.1.8
dashscope                1.22.1
diffusers                0.32.2
easydict                 1.13
einops                   0.8.1
exceptiongroup           1.2.2
fastapi                  0.115.8
ffmpy                    0.5.0
filelock                 3.17.0
flash_attn               2.7.4.post1
frozenlist               1.5.0
fsspec                   2025.2.0
ftfy                     6.3.1
gradio                   5.19.0
gradio_client            1.7.2
h11                      0.14.0
httpcore                 1.0.7
httpx                    0.28.1
huggingface-hub          0.29.1
idna                     3.10
imageio                  2.37.0
imageio-ffmpeg           0.6.0
importlib_metadata       8.6.1
Jinja2                   3.1.5
markdown-it-py           3.0.0
MarkupSafe               2.1.5
mdurl                    0.1.2
modelscope               1.23.1
mpmath                   1.3.0
multidict                6.1.0
networkx                 3.4.2
numpy                    1.26.4
nvidia-cublas-cu12       12.4.5.8
nvidia-cuda-cupti-cu12   12.4.127
nvidia-cuda-nvrtc-cu12   12.4.127
nvidia-cuda-runtime-cu12 12.4.127
nvidia-cudnn-cu12        9.1.0.70
nvidia-cufft-cu12        11.2.1.3
nvidia-curand-cu12       10.3.5.147
nvidia-cusolver-cu12     11.6.1.9
nvidia-cusparse-cu12     12.3.1.170
nvidia-cusparselt-cu12   0.6.2
nvidia-nccl-cu12         2.21.5
nvidia-nvjitlink-cu12    12.4.127
nvidia-nvtx-cu12         12.4.127
opencv-python            4.11.0.86
orjson                   3.10.15
packaging                24.2
pandas                   2.2.3
pillow                   11.1.0
pip                      25.0
propcache                0.3.0
psutil                   7.0.0
pydantic                 2.10.6
pydantic_core            2.27.2
pydub                    0.25.1
Pygments                 2.19.1
python-dateutil          2.9.0.post0
python-multipart         0.0.20
pytz                     2025.1
PyYAML                   6.0.2
regex                    2024.11.6
requests                 2.32.3
rich                     13.9.4
ruff                     0.9.7
safehttpx                0.1.6
safetensors              0.5.3
semantic-version         2.10.0
setuptools               75.8.0
shellingham              1.5.4
six                      1.17.0
sniffio                  1.3.1
starlette                0.45.3
sympy                    1.13.1
tokenizers               0.21.0
tomlkit                  0.13.2
torch                    2.6.0
torchvision              0.21.0
tqdm                     4.67.1
transformers             4.49.0
triton                   3.2.0
typer                    0.15.1
typing_extensions        4.12.2
tzdata                   2025.1
urllib3                  2.3.0
uvicorn                  0.34.0
wcwidth                  0.2.13
websocket-client         1.8.0
websockets               15.0
wheel                    0.45.1
yarl                     1.18.3
zipp                     3.21.0
(wan) ubuntu@ubuntu-server:~$ 

说明：

在安装flash_attn时，可能出现错误，多次尝试也不能完成。可以先在requirements.txt中先注释掉flash_attn，安装完成后，再单独安装flash_attn即可。

模型下载

下载地址

https://www.modelscope.cn/models/Wan-AI/Wan2.1-T2V-1.3B

使用modelscope客户端下载模型文件

pip install modelscopecd /home/ubuntu/code/Wan2.1modelscope download --model Wan-AI/Wan2.1-T2V-1.3B README.md --local_dir ./Wan2.1-T2V-1.3B

生成视频

官网的视频生成例子

python generate.py  --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --sample_shift 8 --sample_guide_scale 6 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."

或

python generate.py  --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --offload_model True --t5_cpu --sample_shift 8 --sample_guide_scale 6 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."

演示如下：

(wan) ubuntu@ubuntu-server:~/code/Wan2.1$ export CUDA_VISIBLE_DEVICES=1
(wan) ubuntu@ubuntu-server:~/code/Wan2.1$ python generate.py  --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --sample_shift 8 --sample_guide_scale 6 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."
[2025-02-28 09:44:02,400] INFO: offload_model is not specified, set to True.
[2025-02-28 09:44:02,400] INFO: Generation job args: Namespace(task='t2v-1.3B', size='832*480', frame_num=81, ckpt_dir='./Wan2.1-T2V-1.3B', offload_model=True, ulysses_size=1, ring_size=1, t5_fsdp=False, t5_cpu=False, dit_fsdp=False, save_file=None, prompt='Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage.', use_prompt_extend=False, prompt_extend_method='local_qwen', prompt_extend_model=None, prompt_extend_target_lang='ch', base_seed=1470089367940419568, image=None, sample_solver='unipc', sample_steps=50, sample_shift=8.0, sample_guide_scale=6.0)
[2025-02-28 09:44:02,400] INFO: Generation model config: {'__name__': 'Config: Wan T2V 1.3B', 't5_model': 'umt5_xxl', 't5_dtype': torch.bfloat16, 'text_len': 512, 'param_dtype': torch.bfloat16, 'num_train_timesteps': 1000, 'sample_fps': 16, 'sample_neg_prompt': '色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量 ，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走', 't5_checkpoint': 'models_t5_umt5-xxl-enc-bf16.pth', 't5_tokenizer': 'google/umt5-xxl', 'vae_checkpoint': 'Wan2.1_VAE.pth', 'vae_stride': (4, 8, 8), 'patch_size': (1, 2, 2), 'dim': 1536, 'ffn_dim': 8960, 'freq_dim': 256, 'num_heads': 12, 'num_layers': 30, 'window_size': (-1, -1), 'qk_norm': True, 'cross_attn_norm': True, 'eps': 1e-06}
[2025-02-28 09:44:02,400] INFO: Input prompt: Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage.
[2025-02-28 09:44:02,400] INFO: Creating WanT2V pipeline.

视频生成时，大约占用20G的显存空间。

用时4分12秒，完成视频的生成。

视频内容如下：

CSDN对视频审核不过，不知道为什么，以gif显示。

截图如下：

简单描述场景视频生成示例

简单描述一个场景，使用百度翻译为英文。

一个人晚上开着保时捷在城市的道路上，周围是五光十色的广告和灯光。摄像头是驾驶员位置，光圈4.5，用于虚化周围的楼房和行人，汽车等。A person is driving a Porsche on the city road at night, surrounded by colorful advertisements and lights. The camera is located at the driver's position with an aperture of 4.5, used to blur surrounding buildings, pedestrians, cars, etc.

运行视频生成程序

python generate.py  --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --sample_shift 8 --sample_guide_scale 6 --prompt "A person is driving a Porsche on the city road at night, surrounded by colorful advertisements and lights. The camera is located at the driver's position with an aperture of 4.5, used to blur surrounding buildings, pedestrians, cars, etc."

大概4分钟后，生成视频文件。

视频生成文件如下：

CSDN对视频审核不过，不知道为什么，使用GIF显示。

截图如下：

详细描述场景视频生成示例

使用大模型生成视频描述，加入更多细节。

使用deepseek，扩写生成视频的描述文字。

将下面的场景内容扩写到500字，加入更多的细节描述，并保持场景不变，用于视频生成的描述。场景内容如下：一个人晚上开着保时捷在城市的道路上，周围是五光十色的广告和灯光。摄像头是驾驶员位置，光圈4.5，用于虚化周围的楼房和行人，汽车等。

生成内容如下：

【视频场景描述 - 夜驰光影】镜头以第一人称视角透过保时捷911 Carrera的挡风玻璃展开。仪表盘泛着宝石蓝的冷光，数字时速表在87km/h轻微震颤，转速表红色指针在4000转区域规律摆动。驾驶者戴着黑色鳄鱼皮手套的双手虚搭在Alcantara麂皮方向盘三点九点位置，左腕的铂金腕表在仪表背光中偶尔闪过金属寒光。车窗外，霓虹光流在光圈f/4.5的虚化下融化成斑斓的星轨。右侧商业街的LED巨幕正轮播着香水广告，克里斯汀·斯图尔特的面容在动态模糊中化作粉色与银白的光瀑。后视镜里，追逐的出租车顶灯在浅景深中晕染成跳动的朱红色光斑，如同都市丛林里躁动的萤火虫群。皮革缝线座椅随着过弯微微变形，能清晰看见Nappa真皮表面细小的毛孔纹路。空调出风口飘散着雪松香氛，混着方向盘后侧杯架里半杯威士忌的泥煤气息。转速突然攀升时，水平对置六缸引擎的低吼透过碳纤维车架传来，像远处闷雷滚过云层。22:17分，驶入跨江隧道。隧道壁的条形灯在高速运动中拉成金色光带，顶棚通风口的气流将驾驶者额前碎发吹起，露出被阴影覆盖的眉骨。仪表盘亮度自动调暗的瞬间，后车远光灯在前挡风玻璃上折射出彩虹光晕，如同在暴雨后的挡风玻璃上划开一道棱镜。驶出隧道的刹那，挡风玻璃突然映满璀璨星河——实为金融区摩天楼的玻璃幕墙群，每扇窗户都化作发光的矩阵单元。左侧在建大厦的塔吊亮着警示红灯，在长曝光效果中拖曳出妖冶的光弧。驾驶者左手无意识地摩挲着门把手上保时捷盾徽的浮雕纹路，右手食指在换挡拨片留下转瞬即逝的指纹。特写镜头掠过转速表突然跃升的红色指针，挡风玻璃倒影中可见驾驶者黑色衬衫袖口露出1厘米的苍白手腕。后视镜挂着的平安符在加速度中扬起45度角，金线刺绣的"安全"二字在虚焦中化作两团跃动的光晕。此刻仪表台时钟跳至22:43，车载屏幕导航显示正驶向跨海大桥方向，地图上的蓝色路径线在黑夜中如一道激光切开霓虹海洋。

使用百度翻译，将提示词翻译为英文，并进行视频生成命令。

python generate.py  --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --sample_shift 8 --sample_guide_scale 6 --prompt "The camera unfolds through the windshield of the Porsche 911 Carrera from a first person perspective. The dashboard emits a cool gemstone blue light, the digital speedometer vibrates slightly at 87km/h, and the red pointer of the tachometer swings regularly in the 4000 rpm range. The driver's hands, wearing black crocodile leather gloves, were loosely resting on the Alcantara suede steering wheel at the 3:09 o'clock position, and the platinum wristwatch on the left wrist occasionally flashed a metallic cold light in the instrument backlight. Outside the car window, the neon light flow melts into a colorful star track under the blurring of the aperture f/4.5. The LED giant screen on the right side of the commercial street is broadcasting advertisements for perfume. Kristen Stewart's face turns into pink and silver waterfall in dynamic blur. In the rearview mirror, the chasing taxi's overhead lights blend into a pulsating crimson light spot in the shallow depth of field, like a restless firefly swarm in the urban jungle. The leather stitched seat slightly deforms as it bends, and the fine pore patterns on the Nappa leather surface can be clearly seen. The air conditioning vent is filled with the fragrance of cedar, mixed with the peat smell of half a glass of whiskey in the cup holder behind the steering wheel. When the engine speed suddenly increased, the low roar of the horizontally opposed six cylinder engine came through the carbon fiber frame, like thunder rolling through the clouds in the distance. At 22:17, enter the cross river tunnel. The strip lights on the tunnel walls create a golden light strip during high-speed movement, and the airflow from the ceiling vents blows up the driver's broken hair in front of their forehead, revealing the shaded brow bones. At the moment when the brightness of the instrument panel automatically dims, the rear high beam lamp refracts a rainbow halo on the front windshield, just like a prism on the windshield after a rainstorm. At the moment of exiting the tunnel, the windshield suddenly reflected a dazzling galaxy - actually a group of glass curtain walls of the financial district skyscraper, with each window turning into a luminous matrix unit. The tower crane of the building under construction on the left is displaying a warning red light, dragging a seductive arc of light in the long exposure effect. The driver unconsciously rubbed the relief pattern of the Porsche emblem on the door handle with their left hand, and left a fleeting fingerprint on the shift paddles with their right index finger. A close-up shot passes by the red pointer on the tachometer that suddenly jumps, revealing a pale wrist of 1 centimeter on the driver's black shirt sleeve in the reflection of the windshield. The safety symbol hanging on the rearview mirror rises at a 45 degree angle in acceleration, and the gold embroidered word "safety" turns into two jumping halos in virtual focus. At this moment, the dashboard clock jumps to 22:43, and the car screen navigation shows that it is heading towards the direction of the cross sea bridge. The blue path line on the map cuts through the neon ocean like a laser in the dark night."

视频生成文件：

CSDN对视频审核不过，不知道为什么，用GIF显示。

截图如下