mlsanityData to 7 Oct 2026

MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model

arXiv 2208.15001 ↗First posted 31 Aug 2022cs.CVcited by 768

On the map: Diffusion models → Video generation

Abstract (arXiv)

Human motion modeling is important for many modern graphics applications, which typically require professional skills. In order to remove the skill barriers for laymen, recent motion generation methods can directly generate human motions conditioned on natural languages. However, it remains challenging to achieve diverse and fine-grained motion generation with various text inputs. To address this problem, we propose MotionDiffuse, the first diffusion model-based text-driven motion generation framework, which demonstrates several desired properties over existing methods. 1) Probabilistic Mapping. Instead of a deterministic language-motion mapping, MotionDiffuse generates motions through a series of denoising steps in which variations are injected. 2) Realistic Synthesis. MotionDiffuse excels at modeling complicated data distribution and generating vivid motion sequences. 3) Multi-Level Manipulation. MotionDiffuse responds to fine-grained instructions on body parts, and arbitrary-length motion synthesis with time-varied text prompts. Our experiments show MotionDiffuse outperforms existing SoTA methods by convincing margins on text-driven motion generation and action-conditioned motion generation. A qualitative analysis further demonstrates MotionDiffuse's controllability for comprehensive motion generation. Homepage: https://mingyuan-zhang.github.io/projects/MotionDiffuse.html

Key facts. Cited by 768 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 49 of its 90 references that are in the map. First posted on arXiv 31 Aug 2022.

Builds on

49 of 90 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  7. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  8. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  9. 2016Density estimation using Real NVPcited by 2,029
  10. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  11. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  12. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
37 more
  1. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  2. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  3. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 530
  4. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  5. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 497
  6. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  7. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
  8. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 479
  9. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  10. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  11. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 438
  12. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 406
  13. 2018Efficient Attention: Attention with Linear Complexitiescited by 366
  14. 2016The KIT Motion-Language Datasetcited by 334
  15. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
  16. 2021BABEL: Bodies, Action and Behavior with English Labelscited by 261
  17. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  18. 2020FineGym: A Hierarchical Video Dataset for Fine-grained Action Understandingcited by 232
  19. 2021Robust Motion In-betweeningcited by 223
  20. 2018Moments in Time Dataset: one million videos for event understandingcited by 215
  21. 2022Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memorycited by 210
  22. 2021AGORA: Avatars in Geography Optimized for Regression Analysiscited by 203
  23. 2020Long-term Human Motion Prediction with Scene Contextcited by 191
  24. 2021Synthesis of Compositional Animations from Textual Descriptionscited by 177
  25. 2017HP-GAN: Probabilistic 3D human motion prediction via GANcited by 176
  26. 2016Panoptic Studio: A Massively Multiview System for Social Interaction Capturecited by 173
  27. 2017PoseTrack: A Benchmark for Human Pose Estimation and Trackingcited by 173
  28. 2022HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modelingcited by 115
  29. 2021Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agentscited by 106
  30. 2018MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamicscited by 86
  31. 2020Learning to Generate Diverse Dance Motions with Transformercited by 85
  32. 2020Dance Revolution: Long-Term Dance Generation with Music via Curriculum Learningcited by 81
  33. 2021Playing for 3D Human Recoverycited by 63
  34. 2022Implicit Neural Representations for Variable Length Human Motion Generationcited by 61
  35. 2019Learning Diverse Stochastic Human-Action Generators by Learning Smooth Latent Transitionscited by 45
  36. 2020HAA500: Human-Centric Atomic Action Dataset with Curated Videoscited by 38
  37. 20203D Human Shape Reconstruction from a Polarization Imagecited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 21; 2023: 140; 2024: 207; 2025: 231; 2026: 169.

768 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  2. 2022Human Motion Diffusion Modelcited by 1,062
  3. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
  4. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
  5. 2023MotionGPT: Human Motion as a Foreign Languagecited by 521
  6. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 388
  7. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 365
  8. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  9. 2023Human Motion Diffusion as a Generative Priorcited by 354
  10. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  11. 2022EDGE: Editable Dance Generation From Musiccited by 284
  12. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 282
38 more
  1. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 237
  2. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 233
  3. 2023Guided Motion Diffusion for Controllable Human Motion Synthesiscited by 221
  4. 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 220
  5. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  6. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
  7. 2023Object Motion Guided Human Motion Synthesiscited by 206
  8. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 205
  9. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Nowcited by 204
  10. 2023TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesiscited by 194
  11. 2022Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Modelscited by 188
  12. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 181
  13. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 167
  14. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 164
  15. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Modelscited by 164
  16. 2023GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latentscited by 156
  17. 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 154
  18. 2023Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuningcited by 150
  19. 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
  20. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 137
  21. 2023DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animationcited by 136
  22. 2023MMM: Generative Masked Motion Modelcited by 129
  23. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  24. 2023AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanismcited by 116
  25. 2024Motion Mamba: Efficient and Long Sequence Motion Generationcited by 111
  26. 2024Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wildcited by 109
  27. 2023Controllable Human-Object Interaction Synthesiscited by 107
  28. 2023FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editingcited by 105
  29. 2023Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Modelcited by 104
  30. 2023HumanTOMATO: Text-aligned Whole-body Motion Generationcited by 103
  31. 2023Inter-X: Towards Versatile Human-Human Interaction Analysiscited by 98
  32. 2023Optimizing Diffusion Noise Can Serve As Universal Motion Priorscited by 97
  33. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 94
  34. 2022Ego-Body Pose Estimation via Ego-Head Pose Estimationcited by 93
  35. 2023TLControl: Trajectory and Language Control for Human Motion Synthesiscited by 93
  36. 2023Unified Human-Scene Interaction via Prompted Chain-of-Contactscited by 91
  37. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 89
  38. 2024MotionLLM: Understanding Human Behaviors from Human Motions and Videoscited by 89

50 most cited of 768 All 768 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Coordinating Multiple Conditions for Trajectory-Controlled Human Motion Generation
  2. 2024Towards motion from video diffusion models
  3. 2022Diffusion Motion: Generate Text-Guided 3D Human Motion by Diffusion Model
  4. 2025Strong and Controllable 3D Motion Generation
  5. 2025PackDiT: Joint Human Motion and Text Generation via Mutual Prompting
  6. 2022FLAME: Free-form Language-based Motion Synthesis & Editing
  7. 2023AMD: Autoregressive Motion Diffusion
  8. 2024FTMoMamba: Motion Generation with Frequency and Text State Space Models