mlsanityData to 14 Oct 2026

Guided Motion Diffusion for Controllable Human Motion Synthesis

arXiv 2305.12577 ↗First posted 21 May 2023cs.CVcited by 223

On the map: Diffusion models → Video generation

Abstract (arXiv)

Denoising diffusion models have shown great promise in human motion synthesis conditioned on natural language descriptions. However, integrating spatial constraints, such as pre-defined motion trajectories and obstacles, remains a challenge despite being essential for bridging the gap between isolated human motion and its surrounding environment. To address this issue, we propose Guided Motion Diffusion (GMD), a method that incorporates spatial constraints into the motion generation process. Specifically, we propose an effective feature projection scheme that manipulates motion representation to enhance the coherency between spatial information and local poses. Together with a new imputation formulation, the generated motion can reliably conform to spatial constraints such as global motion trajectories. Furthermore, given sparse spatial constraints (e.g. sparse keyframes), we introduce a new dense guidance approach to turn a sparse signal, which is susceptible to being ignored during the reverse steps, into denser signals to guide the generated motion to the given constraints. Our extensive experiments justify the development of GMD, which achieves a significant improvement over state-of-the-art methods in text-based motion generation while allowing control of the synthesized motions with spatial constraints.

Key facts. Cited by 223 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 61 of its 69 references that are in the map. First posted on arXiv 21 May 2023.

Builds on

61 of 69 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023Modiff: Action-Conditioned 3D Motion Generation with Denoising Diffusion Probabilistic Models10 Jan
  2. 2022Executing your Commands via Motion Diffusion in Latent Space8 Dec
  3. 2022Pretrained Diffusion Models for Unified Human Motion Synthesis6 Dec
  4. 2022PhysDiff: Physics-Guided Human Motion Diffusion Model5 Dec
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct
  6. 2022Human Motion Diffusion Model29 Sep
  7. 2022FLAME: Free-form Language-based Motion Synthesis & Editing1 Sep
  8. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model31 Aug
  9. 2022Video Diffusion Models7 Apr

Other references · 52

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2022Scalable Diffusion Models with Transformerscited by 6,054
  11. 2022Classifier-Free Diffusion Guidancecited by 5,847
  12. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
38 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  2. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  3. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  4. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  5. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  6. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  7. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  8. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  9. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  10. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  11. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
  12. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraintscited by 508
  13. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  14. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 492
  15. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 485
  16. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 449
  17. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 447
  18. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  19. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Modelscited by 415
  20. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
  21. 2022Diffusion Models already have a Semantic Latent Spacecited by 328
  22. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 324
  23. 2022Novel View Synthesis with Diffusion Modelscited by 300
  24. 2022EDGE: Editable Dance Generation From Musiccited by 287
  25. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  26. 2021Robust Motion In-betweeningcited by 229
  27. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  28. 2022Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Modelscited by 190
  29. 2018RUDDER: Return Decomposition for Delayed Rewardscited by 173
  30. 2020Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenescited by 135
  31. 2021DanceFormer: Music Conditioned 3D Dance Generation with Parametric Motion Transformercited by 111
  32. 2021Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agentscited by 106
  33. 2021Learning Motion Priors for 4D Human Body Capture in 3D Scenescited by 98
  34. 2021SAGA: Stochastic Whole-Body Grasping with Contactcited by 91
  35. 2020Convolutional Autoencoders for Human Motion Infillingcited by 73
  36. 2022UDE: A Unified Driving Engine for Human Motion Generationcited by 67
  37. 2021Single-Shot Motion Completion with Transformercited by 59
  38. 2020Perpetual Motion: Generating Unbounded Human Motioncited by 36

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 15; 2024: 61; 2025: 82; 2026: 65.

223 papers.

Sort:

By area:

  1. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 168
  2. 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 165
  3. 2024Scaling Up Dynamic Human-Scene Interaction Modelingcited by 137
  4. 2024Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wildcited by 114
  5. 2023Optimizing Diffusion Noise Can Serve As Universal Motion Priorscited by 98
  6. 2023TLControl: Trajectory and Language Control for Human Motion Synthesiscited by 94
  7. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 93
  8. 2024Multi-Track Timeline Control for Text-Driven 3D Human Motion Generationcited by 91
  9. 2023EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generationcited by 89
  10. 2024Generating Human Interaction Motions in Scenes with Text Controlcited by 80
  11. 2024Flexible Motion In-betweening with Diffusion Modelscited by 71
  12. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 68
38 more
  1. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  2. 2024MotionFix: Text-Driven 3D Human Motion Editingcited by 64
  3. 2023AvatarGPT: All-in-One Framework for Motion Understanding, Planning, Generation and Beyondcited by 63
  4. 2023ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactionscited by 62
  5. 2024DiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptionscited by 62
  6. 2024Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitivescited by 61
  7. 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understandingcited by 59
  8. 2024Estimating Body and Hand Motion in an Ego-sensed Worldcited by 58
  9. 2024Taming Diffusion Probabilistic Models for Character Controlcited by 57
  10. 2023Interactive Character Control with Auto-Regressive Motion Diffusion Modelscited by 55
  11. 2024ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Modelcited by 53
  12. 2024Generating Human Motion in 3D Scenes from Text Descriptionscited by 49
  13. 2024RoHM: Robust Human Motion Reconstruction via Diffusioncited by 48
  14. 2024The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motioncited by 46
  15. 2023I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactionscited by 44
  16. 2024G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesiscited by 43
  17. 2023Fast Diffusion-Based Counterfactuals for Shortcut Removal and Generationcited by 41
  18. 2024MoGenTS: Motion Generation based on Spatial-Temporal Joint Modelingcited by 41
  19. 2025AnyTop: Character Animation Diffusion with Any Topologycited by 41
  20. 2025SALAD: Skeleton-aware Latent Diffusion for Text-driven Motion Generation and Editingcited by 39
  21. 2024MaskControl: Spatio-Temporal Control for Masked Motion Synthesiscited by 38
  22. 2026Kimodo: Scaling Controllable Human Motion Generationcited by 37
  23. 2024HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objectscited by 36
  24. 2024Enabling Synergistic Full-Body Control in Prompt-Based Co-Speech Motion Generationcited by 35
  25. 2024Monkey See, Monkey Do: Harnessing Self-attention in Motion Diffusion for Zero-shot Motion Transfercited by 31
  26. 2024MotionChain: Conversational Motion Controllers via Multimodal Promptscited by 30
  27. 2025MotionLab: Unified Human Motion Generation and Editing via the Motion-Condition-Motion Paradigmcited by 30
  28. 2025Absolute Coordinates Make Motion Generation Easycited by 29
  29. 2025Diffuse-CLoC: Guided Diffusion for Physics-based Character Look-ahead Controlcited by 28
  30. 2025PARC: Physics-based Augmentation with Reinforcement Learning for Character Controllerscited by 28
  31. 2024ReinDiffuse: Crafting Physically Plausible Motions with Reinforced Diffusion Modelcited by 26
  32. 2025A Survey on Human Interaction Motion Generationcited by 25
  33. 2025UniPhys: Unified Planner and Controller with Diffusion for Flexible Physics-Based Character Controlcited by 25
  34. 2025Guiding Human-Object Interactions with Rich Geometry and Relationscited by 24
  35. 2023InterControl: Zero-shot Human Interaction Generation by Controlling Every Jointcited by 23
  36. 2025Fg-T2M++: LLMs-Augmented Fine-Grained Text Driven Human Motion Generationcited by 21
  37. 2024Programmable Motion Generation for Open-Set Motion Control Taskscited by 20
  38. 2025ChainHOI: Joint-based Kinematic Chain Modeling for Human-Object Interaction Generationcited by 20

50 most cited of 223 All 223 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Diffusion Motion: Generate Text-Guided 3D Human Motion by Diffusion Model
  2. 2023DiffusionPhase: Motion Diffusion in Frequency Domain
  3. 2024MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation
  4. 2026Coordinating Multiple Conditions for Trajectory-Controlled Human Motion Generation
  5. 2026Diffusion Path Alignment for Long-Range Motion Generation and Domain Transitions
  6. 2023Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model
  7. 2023BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis
  8. 2022Unifying Human Motion Synthesis and Style Transfer with Denoising Diffusion Probabilistic Models