Reference-to-video

Wan2.1-VACE

VACE (Video All-in-one Creation and Editing, ICCV 2025, arXiv:2503.07598) wraps editing, reference images, and masks into a Video Condition Unit, then injects them through a Context Adapter on the Wan DiT. Weights: Wan2.1-VACE-1.3B (~81×480×832) and Wan2.1-VACE-14B (~81×720×1280), Apache 2.0.

• R2V — no input video; concepts come from text + reference images.
• V2V — pixel-level restyle / structure / motion preservation on a video.
• MV2V — masked local edit (inpaint, swap, expand).
• Compositions: Reference Anything, Move Anything, Animate Anything, Swap Anything, Expand Anything.

Wan2.2-Animate-14B

Animate is a 2.2-era specialist. Animation mode: the character image copies motion and expression from a drive video. Replacement mode: the character is inserted into the drive video (mask + background + optional relighting LoRA). Preprocess extracts pose/face (and for replace, background/mask) before generate.py --task animate-14B.

python generate.py --task animate-14B --ckpt_dir ./Wan2.2-Animate-14B/ --src_root_path ./examples/wan_animate/animate/process_results/ --refert_num 1

S2V as a reference path

Wan2.2-S2V-14B can take --pose_video so audio and a pose sequence drive the same identity image. That is reference-to-video with speech, not VACE.

FAQ

VACE or Animate?

Need inpaint/outpaint/depth/pose edit of an existing clip → VACE. Need a still character to perform a drive video → Animate. Need talking from audio → S2V.

Is R2V in Wan2.2 T2V?

No extra reference image slot on T2V-A14B. Use VACE R2V or Animate.