mlsanityData to 7 Oct 2026

MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance

arXiv 2406.19680 ↗First posted 28 Jun 2024cs.CV cs.AI cs.MMcited by 196

Not on the map. Why?

Cited by 60 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In recent years, generative artificial intelligence has achieved significant advancements in the field of image generation, spawning a variety of applications. However, video generation still faces considerable challenges in various aspects, such as controllability, video length, and richness of details, which hinder the application and popularization of this technology. In this work, we propose a controllable video generation framework, dubbed MimicMotion, which can generate high-quality videos of arbitrary length mimicking specific motion guidance. Compared with previous methods, our approach has several highlights. Firstly, we introduce confidence-aware pose guidance that ensures high frame quality and temporal smoothness. Secondly, we introduce regional loss amplification based on pose confidence, which significantly reduces image distortion. Lastly, for generating long and smooth videos, we propose a progressive latent fusion strategy. By this means, we can produce videos of arbitrary length with acceptable resource consumption. With extensive experiments and user studies, MimicMotion demonstrates significant improvements over previous approaches in various aspects. Detailed results and comparisons are available on our project page: https://tencent.github.io/MimicMotion .

Key facts. Cited by 196 papers in the map (data through 7 Oct 2026). Builds on 40 of its 52 references that are in the map. First posted on arXiv 28 Jun 2024.

Builds on

40 of 52 references.

References · 40

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  8. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  9. 2022Video Diffusion Modelscited by 2,280
  10. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  11. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  12. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
28 more
  1. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  2. 2020First Order Motion Model for Image Animationcited by 698
  3. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  4. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  5. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
  6. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  7. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  8. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  9. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  10. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 431
  11. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
  12. 2022MAGVIT: Masked Generative Video Transformercited by 341
  13. 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
  14. 2018Everybody Dance Nowcited by 313
  15. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 312
  16. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  17. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
  18. 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 271
  19. 2021Motion Representations for Articulated Animationcited by 253
  20. 2022Thin-Plate Spline Motion Model for Image Animationcited by 209
  21. 2023NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generationcited by 170
  22. 2023DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusioncited by 167
  23. 2024Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Modelscited by 159
  24. 2023DisCo: Disentangled Control for Realistic Human Dance Generationcited by 141
  25. 2023MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusioncited by 115
  26. 2024MagicVideo-V2: Multi-Stage High-Aesthetic Video Generationcited by 48
  27. 2024VividPose: Advancing Stable Video Diffusion for Realistic Human Image Animationcited by 27
  28. 2023DreaMoving: A Human Video Generation Framework based on Diffusion Modelscited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 32; 2025: 106; 2026: 58.

196 papers.

Sort:

By area:

  1. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 202
  2. 2024ControlNeXt: Powerful and Efficient Control for Image and Video Generationcited by 158
  3. 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 127
  4. 2025Aether: Geometric-Aware Unified World Modelingcited by 94
  5. 2025Controllable Video Generation: A Surveycited by 93
  6. 2024Animate-X: Universal Character Image Animation with Enhanced Motion Representationcited by 86
  7. 2024StableAnimator: High-Quality Identity-Preserving Human Image Animationcited by 85
  8. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 84
  9. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
  10. 2025Wan-Animate: Unified Character Animation and Replacement with Holistic Replicationcited by 76
  11. 2024DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generationcited by 73
  12. 2024EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animationcited by 69
38 more
  1. 2024MIMO: Controllable Character Video Synthesis with Spatial Decomposed Modelingcited by 63
  2. 2025Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordancecited by 60
  3. 2025Wan-S2V: Audio-Driven Cinematic Video Generationcited by 60
  4. 2024IDOL: Instant Photorealistic 3D Human Creation from a Single Imagecited by 55
  5. 2025HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animationcited by 55
  6. 2024PhysMotion: Physics-Grounded Dynamics From a Single Imagecited by 45
  7. 2025StreamDiT: Real-Time Streaming Text-to-Video Generationcited by 43
  8. 2025VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Controlcited by 42
  9. 2024Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesiscited by 41
  10. 2025PhysAnimator: Physics-Guided Generative Cartoon Animationcited by 40
  11. 2024LeviTor: 3D Trajectory Oriented Image-to-Video Synthesiscited by 39
  12. 2024CyberHost: Taming Audio-driven Avatar Diffusion Model with Region Codebook Attentioncited by 38
  13. 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generationcited by 38
  14. 2025DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidancecited by 36
  15. 2024DisPose: Disentangling Pose Guidance for Controllable Human Image Animationcited by 35
  16. 2025OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulationcited by 30
  17. 2025RealisDance-DiT: Simple yet Strong Baseline towards Controllable Character Animation in the Wildcited by 28
  18. 2025FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarioscited by 25
  19. 2025Human Motion Video Generation: A Surveycited by 25
  20. 2024A Comprehensive Survey on Human Video Generation: Challenges, Methods, and Insightscited by 24
  21. 2024MotionCraft: Crafting Whole-Body Motion with Plug-and-Play Multimodal Controlscited by 24
  22. 2024Repurposing Pre-trained Video Diffusion Models for Event-based Video Interpolationcited by 23
  23. 2024Generative Inbetweening through Frame-wise Conditions-Driven Video Generationcited by 23
  24. 2025What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewardscited by 23
  25. 2025OmniVDiff: Omni Controllable Video Diffusion for Generation and Understandingcited by 22
  26. 2025GUAVA: Generalizable Upper Body 3D Gaussian Avatarcited by 22
  27. 2025InterActHuman: Multi-Concept Human Animation with Layout-Aligned Audio Conditionscited by 20
  28. 2025HumanDreamer: Generating Controllable Human-Motion Videos via Decoupled Generationcited by 19
  29. 2025DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformerscited by 19
  30. 2025X-Dyna: Expressive Dynamic Human Image Animationcited by 18
  31. 2025OmniV2V: Versatile Video Generation and Editing via Dynamic Content Manipulationcited by 18
  32. 2025Generative AI for Cel-Animation: A Surveycited by 17
  33. 2025Reasoning Physical Video Generation with Diffusion Timestep Tokens via Reinforcement Learningcited by 17
  34. 2024UniVST: A Unified Framework for Training-free Localized Video Style Transfercited by 16
  35. 2024EmotiveTalk: Expressive Talking Head Generation through Audio Information Decoupling and Emotional Video Diffusioncited by 16
  36. 2024DreamDance: Animating Human Images by Enriching 3D Geometry Cues from 2D Posescited by 16
  37. 2025Hallo4: High-Fidelity Dynamic Portrait Animation via Direct Preference Optimizationcited by 16
  38. 2024PERSE: Personalized 3D Generative Avatars from A Single Portraitcited by 15

50 most cited of 196 All 196 (JSON)