mlsanityData to 7 Oct 2026

Human Motion Diffusion as a Generative Prior

arXiv 2303.01418 ↗First posted 2 Mar 2023cs.CV cs.GRcited by 354

On the map: Diffusion models → Video generation

Abstract (arXiv)

Recent work has demonstrated the significant potential of denoising diffusion models for generating human motion, including text-to-motion capabilities. However, these methods are restricted by the paucity of annotated motion data, a focus on single-person motions, and a lack of detailed control. In this paper, we introduce three forms of composition based on diffusion priors: sequential, parallel, and model composition. Using sequential composition, we tackle the challenge of long sequence generation. We introduce DoubleTake, an inference-time method with which we generate long animations consisting of sequences of prompted intervals and their transitions, using a prior trained only for short clips. Using parallel composition, we show promising steps toward two-person generation. Beginning with two fixed priors as well as a few two-person training examples, we learn a slim communication block, ComMDM, to coordinate interaction between the two resulting motions. Lastly, using model composition, we first train individual priors to complete motions that realize a prescribed motion for a given joint. We then introduce DiffusionBlending, an interpolation mechanism to effectively blend several such models to enable flexible and efficient fine-grained joint and trajectory-level control and editing. We evaluate the composition methods using an off-the-shelf motion diffusion model, and further compare the results to dedicated models trained for these specific tasks.

Key facts. Cited by 354 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 33 of its 47 references that are in the map. First posted on arXiv 2 Mar 2023.

Builds on

33 of 47 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Single Motion Diffusion12 Feb
  2. 2022Executing your Commands via Motion Diffusion in Latent Space8 Dec
  3. 2022PhysDiff: Physics-Guided Human Motion Diffusion Model5 Dec
  4. 2022Human Motion Diffusion Model29 Sep
  5. 2022FLAME: Free-form Language-based Motion Synthesis & Editing1 Sep
  6. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model31 Aug

Other references · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2022Classifier-Free Diffusion Guidancecited by 5,717
  8. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  9. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  10. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  11. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  12. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
15 more
  1. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  2. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
  3. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 406
  4. 2022EDGE: Editable Dance Generation From Musiccited by 284
  5. 2021BABEL: Bodies, Action and Behavior with English Labelscited by 261
  6. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  7. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  8. 2022TEACH: Temporal Action Composition for 3D Humanscited by 173
  9. 2022Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fieldscited by 85
  10. 2021Multi-Person 3D Motion Prediction with Multi-Range Transformerscited by 63
  11. 2022MoDi: Unconditional Motion Synthesis from Diverse Datacited by 59
  12. 2021StyleAlign: Analysis and Applications of Aligned StyleGAN Modelscited by 48
  13. 2022Weakly-supervised Action Transition Learning for Stochastic Human Motion Predictioncited by 26
  14. 2022SoMoFormer: Multi-Person Pose Forecasting with Transformerscited by 23
  15. 2022NEURAL MARIONETTE: A Transformer-based Multi-action Human Motion Synthesis Systemcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 41; 2024: 112; 2025: 124; 2026: 77.

354 papers.

Sort:

By area:

  1. 2023MotionGPT: Human Motion as a Foreign Languagecited by 521
  2. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  3. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 233
  4. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 205
  5. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 181
  6. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 164
  7. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 137
  8. 2024Scaling Up Dynamic Human-Scene Interaction Modelingcited by 137
  9. 2023MMM: Generative Masked Motion Modelcited by 129
  10. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  11. 2023TADA! Text to Animatable Digital Avatarscited by 109
  12. 2024Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wildcited by 109
38 more
  1. 2023Controllable Human-Object Interaction Synthesiscited by 107
  2. 2023FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editingcited by 105
  3. 2024CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character controlcited by 103
  4. 2023Inter-X: Towards Versatile Human-Human Interaction Analysiscited by 98
  5. 2023Optimizing Diffusion Noise Can Serve As Universal Motion Priorscited by 97
  6. 2023TLControl: Trajectory and Language Control for Human Motion Synthesiscited by 93
  7. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 89
  8. 2024Multi-Track Timeline Control for Text-Driven 3D Human Motion Generationcited by 88
  9. 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 82
  10. 2024Generating Human Interaction Motions in Scenes with Text Controlcited by 80
  11. 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 80
  12. 2023NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesiscited by 78
  13. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 78
  14. 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 77
  15. 2024Flexible Motion In-betweening with Diffusion Modelscited by 69
  16. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 67
  17. 2024CoMo: Controllable Motion Generation through Language Guided Pose Code Editingcited by 66
  18. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  19. 2023AvatarGPT: All-in-One Framework for Motion Understanding, Planning, Generation and Beyondcited by 63
  20. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 63
  21. 2024MotionFix: Text-Driven 3D Human Motion Editingcited by 63
  22. 2023ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactionscited by 62
  23. 2024Taming Diffusion Probabilistic Models for Character Controlcited by 56
  24. 2024Estimating Body and Hand Motion in an Ego-sensed Worldcited by 56
  25. 2024ReGenNet: Towards Human Action-Reaction Synthesiscited by 53
  26. 2023Interactive Character Control with Auto-Regressive Motion Diffusion Modelscited by 52
  27. 2024ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Modelcited by 52
  28. 2024ParCo: Part-Coordinating Text-to-Motion Synthesiscited by 51
  29. 2024Generating Human Motion in 3D Scenes from Text Descriptionscited by 49
  30. 2023TEDi: Temporally-Entangled Diffusion for Long-Term Motion Synthesiscited by 48
  31. 2024StableMoFusion: Towards Robust and Efficient Diffusion-based Motion Generation Frameworkcited by 48
  32. 2023OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllerscited by 47
  33. 2024InterMask: 3D Human Interaction Generation via Collaborative Masked Modelingcited by 47
  34. 2024RoHM: Robust Human Motion Reconstruction via Diffusioncited by 46
  35. 2024THOR: Text to Human-Object Interaction Diffusion via Relation Interventioncited by 46
  36. 2024Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompanimentcited by 46
  37. 2024Motion-Agent: A Conversational Framework for Human Motion Generation with LLMscited by 46
  38. 2024The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motioncited by 46

50 most cited of 354 All 354 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Synthesizing Long-Term Human Motions with Diffusion Models via Coherent Sampling
  2. 2023DiffusionPhase: Motion Diffusion in Frequency Domain
  3. 2024Generation of Complex 3D Human Motion by Temporal and Spatial Composition of Diffusion Models
  4. 2024MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation
  5. 2023AMD: Autoregressive Motion Diffusion
  6. 2025Back to Basics: Motion Representation Matters for Human Motion Generation Using Diffusion Model
  7. 2026From Diffusion to Flow: Efficient Motion Generation in MotionGPT3
  8. 2025PackDiT: Joint Human Motion and Text Generation via Mutual Prompting