# Diffusion Motion: Generate Text-Guided 3D Human Motion by Diffusion Model

arXiv: https://arxiv.org/abs/2210.12315 · HTML: https://mlsanity.com/paper/2210.12315/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 18 of its 22 references that are in the map.
- First posted on arXiv 22 Oct 2022.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017)
- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [Action2Motion: Conditioned Generation of 3D Human Motions](https://mlsanity.com/paper/2007.15240/) (2020)
- [The KIT Motion-Language Dataset](https://mlsanity.com/paper/1607.03827/) (2016)
- [Language2Pose: Natural Language Grounded Pose Forecasting](https://mlsanity.com/paper/1907.01108/) (2019)
- [Synthesis of Compositional Animations from Textual Descriptions](https://mlsanity.com/paper/2103.14675/) (2021)
- [Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agents](https://mlsanity.com/paper/2101.11101/) (2021)
- [Deep Video Generation, Prediction and Completion of Human Action Sequences](https://mlsanity.com/paper/1711.08682/) (2017)

## Built on by (most cited first, 30 of 41)

- [PhysDiff: Physics-Guided Human Motion Diffusion Model](https://mlsanity.com/paper/2212.02500/) (2022)
- [CG-HOI: Contact-Guided 3D Human-Object Interaction Generation](https://mlsanity.com/paper/2311.16097/) (2023)
- [DiffuseStyleGesture: Stylized Audio-Driven Co-Speech Gesture Generation with Diffusion Models](https://mlsanity.com/paper/2305.04919/) (2023)
- [DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Models](https://mlsanity.com/paper/2310.00434/) (2023)
- [FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models](https://mlsanity.com/paper/2312.08459/) (2023)
- [THOR: Text to Human-Object Interaction Diffusion via Relation Intervention](https://mlsanity.com/paper/2403.11208/) (2024)
- [AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agents](https://mlsanity.com/paper/2403.12835/) (2024)
- [Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking](https://mlsanity.com/paper/2401.06614/) (2024)
- [DINAR: Diffusion Inpainting of Neural Textures for One-Shot Human Avatars](https://mlsanity.com/paper/2303.09375/) (2023)
- [NAP: Neural 3D Articulation Prior](https://mlsanity.com/paper/2305.16315/) (2023)
- [ChatGPT-Powered Hierarchical Comparisons for Image Classification](https://mlsanity.com/paper/2311.00206/) (2023)
- [HUMOS: Human Motion Model Conditioned on Body Shape](https://mlsanity.com/paper/2409.03944/) (2024)
- [Controllable Group Choreography using Contrastive Diffusion](https://mlsanity.com/paper/2310.18986/) (2023)
- [Realistic Human Motion Generation with Cross-Diffusion Models](https://mlsanity.com/paper/2312.10993/) (2023)
- [FloodDiffusion: Tailored Diffusion Forcing for Streaming Motion Generation](https://mlsanity.com/paper/2512.03520/) (2025)
- [UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons](https://mlsanity.com/paper/2309.07051/) (2023)
- [3DiFACE: Diffusion-based Speech-driven 3D Facial Animation and Editing](https://mlsanity.com/paper/2312.00870/) (2023)
- [Pixel Motion as Universal Representation for Robot Control](https://mlsanity.com/paper/2505.07817/) (2025)
- [ProbTalk3D: Non-Deterministic Emotion Controllable Speech-Driven 3D Facial Animation Synthesis Using VQ-VAE](https://mlsanity.com/paper/2409.07966/) (2024)
- [BAD: Bidirectional Auto-regressive Diffusion for Text-to-Motion Generation](https://mlsanity.com/paper/2409.10847/) (2024)
- [LongDanceDiff: Long-term Dance Generation with Conditional Diffusion Model](https://mlsanity.com/paper/2308.11945/) (2023)
- [DEEPTalk: Dynamic Emotion Embedding for Probabilistic Speech-Driven 3D Face Animation](https://mlsanity.com/paper/2408.06010/) (2024)
- [Bipartite Graph Diffusion Model for Human Interaction Generation](https://mlsanity.com/paper/2301.10134/) (2023)
- [Shape My Moves: Text-Driven Shape-Aware Synthesis of Human Motions](https://mlsanity.com/paper/2504.03639/) (2025)
- [AVI-Talking: Learning Audio-Visual Instructions for Expressive 3D Talking Face Generation](https://mlsanity.com/paper/2402.16124/) (2024)
- [GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer](https://mlsanity.com/paper/2408.01826/) (2024)
- [Gradient-Free Classifier Guidance for Diffusion Model Sampling](https://mlsanity.com/paper/2411.15393/) (2024)
- [Pixel Motion Diffusion is What We Need for Robot Control](https://mlsanity.com/paper/2509.22652/) (2025)
- [AnimaMimic: Imitating 3D Animation from Video Priors](https://mlsanity.com/paper/2512.14133/) (2025)
- [PersonaGesture: Single-Reference Co-Speech Gesture Personalization for Unseen Speakers](https://mlsanity.com/paper/2605.06064/) (2026)

## Abstract (arXiv)

We propose a simple and novel method for generating 3D human motion from complex natural language sentences, which describe different velocity, direction and composition of all kinds of actions. Different from existing methods that use classical generative architecture, we apply the Denoising Diffusion Probabilistic Model to this task, synthesizing diverse motion results under the guidance of texts. The diffusion model converts white noise into structured 3D motion by a Markov process with a series of denoising steps and is efficiently trained by optimizing a variational lower bound. To achieve the goal of text-conditioned image synthesis, we use the classifier-free guidance strategy to fuse text embedding into the model during training. Our experiments demonstrate that our model achieves competitive results on HumanML3D test set quantitatively and can generate more visually natural and diverse examples. We also show with experiments that our model is capable of zero-shot generation of motions for unseen text guidance.