mlsanityData to 14 Oct 2026

UniAnimate-DiT: Human Image Animation with Large-Scale Video Diffusion Transformer

arXiv 2504.11289 ↗First posted 15 Apr 2025cs.CV cs.MMcited by 42

On the map: Diffusion models → Video generation

Abstract (arXiv)

This report presents UniAnimate-DiT, an advanced project that leverages the cutting-edge and powerful capabilities of the open-source Wan2.1 model for consistent human image animation. Specifically, to preserve the robust generative capabilities of the original Wan2.1 model, we implement Low-Rank Adaptation (LoRA) technique to fine-tune a minimal set of parameters, significantly reducing training memory overhead. A lightweight pose encoder consisting of multiple stacked 3D convolutional layers is designed to encode motion information of driving poses. Furthermore, we adopt a simple concatenation operation to integrate the reference appearance into the model and incorporate the pose information of the reference image for enhanced pose alignment. Experimental results show that our approach achieves visually appearing and temporally consistent high-fidelity animations. Trained on 480p (832x480) videos, UniAnimate-DiT demonstrates strong generalization capabilities to seamlessly upscale to 720P (1280x720) during inference. The training and inference code is publicly available at https://github.com/ali-vilab/UniAnimate-DiT.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 14 of its 16 references that are in the map. First posted on arXiv 15 Apr 2025.

Builds on

14 of 16 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Survey27 Mar
  2. 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation3 Jun
  3. 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance21 Mar
  4. 2023A Recipe for Scaling up Text-to-Video Generation with Text-free Videos25 Dec
  5. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation28 Nov
  6. 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model27 Nov
  7. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models7 Nov
  8. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  9. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct

Other references · 5

  1. 2022Scalable Diffusion Models with Transformerscited by 6,054
  2. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  3. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  4. 2024Animate-X: Universal Character Image Animation with Enhanced Motion Representationcited by 88
  5. 2018Pose Guided Human Video Generationcited by 67

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 20; 2026: 22.

42 papers.

Sort:

By area:

  1. 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 98
  2. 2025Wan-Animate: Unified Character Animation and Replacement with Holistic Replicationcited by 80
  3. 2025Stable Video Infinity: Infinite-Length Video Generation with Error Recyclingcited by 73
  4. 2025HunyuanVideo-HOMA: Generic Human-Object Interaction in Multimodal Driven Human Animationcited by 14
  5. 2025DanceTogether! Identity-Preserving Multi-Person Interactive Video Generationcited by 13
  6. 2025X-UniMotion: Animating Human Images with Expressive, Unified and Identity-Agnostic Motion Latentscited by 13
  7. 2025SCAIL: Towards Studio-Grade Character Animation via In-Context Learning of 3D-Consistent Pose Representationscited by 13
  8. 2025AniCrafter: Customizing Realistic Human-Centric Animation via Avatar-Background Conditioning in Video Diffusion Modelscited by 10
  9. 2025Animate-X++: Universal Character Image Animation with Dynamic Backgroundscited by 10
  10. 2025SteadyDancer: Harmonized and Coherent Human Image Animation with First-Frame Preservationcited by 10
  11. 2025PoseGen: In-Context LoRA Finetuning for Pose-Controllable Long Human Video Generationcited by 9
  12. 2025One-to-All Animation: Alignment-Free Character Animation and Image Pose Transfercited by 9
30 more
  1. 2026SCAIL-2: Unifying Controlled Character Animation with End-to-End In-Context Conditioningcited by 8
  2. 2026OmniTransfer: All-in-one Framework for Spatio-temporal Video Transfercited by 6
  3. 2025iDiT-HOI: Inpainting-based Hand Object Interaction Reenactment via Video Diffusion Transformercited by 5
  4. 2026DreamActor-M2: Universal Character Image Animation via Spatiotemporal In-Context Learningcited by 5
  5. 2025Generating Human Motion Videos using a Cascaded Text-to-Video Frameworkcited by 4
  6. 2026Kling-MotionControl Technical Reportcited by 4
  7. 2025MTVCraft: Tokenizing 4D Motion for Arbitrary Character Animationcited by 3
  8. 2026CoDance: An Unbind-Rebind Paradigm for Robust Multi-Subject Animationcited by 3
  9. 2026Making Avatars Interact: Towards Text-Driven Human-Object Interaction for Controllable Talking Avatarscited by 3
  10. 2026EverAnimate: Minute-Scale Human Animation via Latent Flow Restorationcited by 3
  11. 2026Image-to-Video Diffusion: From Foundations to Open Frontierscited by 3
  12. 2025ControlHair: Synergizing Physics Simulator and Video Diffusion for Controllable Dynamic Hair Renderingcited by 2
  13. 2026AHOY! Animatable Humans under Occlusion from YouTube Videos with Gaussian Splatting and Video Diffusion Priorscited by 2
  14. 2026StreamHOI: Interaction-aware Temporal Memory Adaptation for Streaming HOI Video Generationcited by 2
  15. 2025In-Video Instructions: Visual Signals as Generative Controlcited by 1
  16. 2025High-Fidelity and Long-Duration Human Image Animation with Diffusion Transformercited by 1
  17. 2026IM-Animation: An Implicit Motion Representation for Identity-decoupled Character Animationcited by 1
  18. 2026GenHOI: Towards Object-Consistent Hand-Object Interaction with Temporally Balanced and Spatially Selective Object Injectioncited by 1
  19. 2026Towards 3D-Aware Video Diffusion Models: Render-Free Human Motion Control with Mesh Tokenizationcited by 1
  20. 2026AgentHOI: Multi-Agent Reasoning for Human-Object-Interaction Video Generation via Implicit Representation Alignmentcited by 1
  21. 2026LiveAnimate: Stable Long-Form Streaming Human Animation in Real-Timecited by 1
  22. 2025Generating Fit Check Videos with a Handheld Cameracited by 0
  23. 2025Screen, Cache, and Match: A Training-Free Causality-Consistent Reference Frame Framework for Human Animationcited by 0
  24. 2026AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animationcited by 0
  25. 2026ActCam: Zero-Shot Joint Camera and 3D Motion Control for Video Generationcited by 0
  26. 2026Implicit Preference Alignment for Human Image Animationcited by 0
  27. 2026Beyond Skeletons: Learning Animation Directly from Driving Videos with Same2X Training Strategycited by 0
  28. 2026Vorch-IR: Long-Form Unified Multimodal Identity Replacement Video Generationcited by 0
  29. 2026SignRefine: Adapting Foundational Video Models for Sign Language Generationcited by 0
  30. 2026Video Generation Models: A Survey of Post-Training and Alignmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025DynamiCtrl: Rethinking the Basic Structure and the Role of Text for High-quality Human Image Animation
  2. 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generation
  3. 2026MultiAnimate: Pose-Guided Image Animation Made Extensible
  4. 2024TCAN: Animating Human Images with Temporally Consistent Pose Guidance using Diffusion Models
  5. 2025StableAnimator++: Overcoming Pose Misalignment and Face Distortion for Human Image Animation
  6. 2025UniMo: Unifying 2D Video and 3D Human Motion with an Autoregressive Framework
  7. 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models
  8. 2025DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidance