mlsanityData to 7 Oct 2026

M2D2M: Multi-Motion Generation from Text with Discrete Diffusion Models

arXiv 2407.14502 ↗First posted 19 Jul 2024cs.CVcited by 23

On the map: Diffusion models → Video generation

Abstract (arXiv)

We introduce the Multi-Motion Discrete Diffusion Models (M2D2M), a novel approach for human motion generation from textual descriptions of multiple actions, utilizing the strengths of discrete diffusion models. This approach adeptly addresses the challenge of generating multi-motion sequences, ensuring seamless transitions of motions and coherence across a series of actions. The strength of M2D2M lies in its dynamic transition probability within the discrete diffusion model, which adapts transition probabilities based on the proximity between motion tokens, encouraging mixing between different modes. Complemented by a two-phase sampling strategy that includes independent and joint denoising steps, M2D2M effectively generates long-term, smooth, and contextually coherent human motion sequences, utilizing a model trained for single-motion generation. Extensive experiments demonstrate that M2D2M surpasses current state-of-the-art benchmarks for motion generation from text descriptions, showcasing its efficacy in interpreting language semantics and generating dynamic, realistic motions.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 49 of its 66 references that are in the map. First posted on arXiv 19 Jul 2024.

Builds on

49 of 66 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editing22 Dec
  2. 2023Priority-Centric Human Motion Generation in Discrete Latent Space28 Aug
  3. 2023Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation16 May
  4. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model3 Apr
  5. 2023Human Motion Diffusion as a Generative Prior2 Mar
  6. 2022Executing your Commands via Motion Diffusion in Latent Space8 Dec
  7. 2022Human Motion Diffusion Model29 Sep
  8. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model31 Aug

Other references · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Classifier-Free Diffusion Guidancecited by 5,717
  9. 2016Layer Normalizationcited by 5,579
  10. 2017Neural Discrete Representation Learningcited by 4,650
  11. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  12. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
29 more
  1. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  2. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
  3. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  4. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  5. 2023MotionGPT: Human Motion as a Foreign Languagecited by 521
  6. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
  7. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 406
  8. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
  9. 2016The KIT Motion-Language Datasetcited by 334
  10. 2021BABEL: Bodies, Action and Behavior with English Labelscited by 261
  11. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  12. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 245
  13. 2021Robust Motion In-betweeningcited by 223
  14. 2022Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Modelscited by 188
  15. 2021Synthesis of Compositional Animations from Textual Descriptionscited by 177
  16. 2022TEACH: Temporal Action Composition for 3D Humanscited by 173
  17. 2020History Repeats Itself: Human Motion Prediction via Motion Attentioncited by 164
  18. 2023GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latentscited by 156
  19. 2023AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanismcited by 116
  20. 2021ViTGAN: Training GANs with Vision Transformerscited by 110
  21. 2017Text2Action: Generative Adversarial Synthesis from Language to Actioncited by 106
  22. 2021Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agentscited by 106
  23. 2018MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamicscited by 86
  24. 2020Convolutional Autoencoders for Human Motion Infillingcited by 73
  25. 2021Single-Shot Motion Completion with Transformercited by 59
  26. 2018Recurrent Transition Networks for Character Locomotioncited by 52
  27. 2019Analyzing Input and Output Representations for Speech-Driven Gesture Generationcited by 51
  28. 2022MultiAct: Long-Term 3D Human Motion Generation from Multiple Action Labelscited by 45
  29. 2022Real-time Controllable Motion Transition for Characterscited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 10; 2026: 8.

23 papers.

Sort:

By area:

  1. 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 82
  2. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  3. 2024How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Frameworkcited by 48
  4. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  5. 2024The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motioncited by 46
  6. 2024DisCoRD: Discrete Tokens to Continuous Motion via Rectified Flow Decodingcited by 17
  7. 2024InfiniDreamer: Arbitrarily Long Human Motion Generation via Segment Score Distillationcited by 12
  8. 2024Estimating Ego-Body Pose from Doubly Sparse Egocentric Video Datacited by 8
  9. 2025Text-driven Motion Generation: Overview, Challenges and Directionscited by 8
  10. 2025Deep Compositional Phase Diffusion for Long Motion Sequence Generationcited by 5
  11. 2026ActionParty: Multi-Subject Action Binding in Generative Video Gamescited by 4
  12. 2025Human Motion Prediction, Reconstruction, and Generationcited by 1
11 more
  1. 2025Multimodal Generative AI with Autoregressive LLMs for Human Motion Understanding and Generation: A Way Forwardcited by 1
  2. 2025Occlusion-Aware Temporally Consistent Amodal Completion for 3D Human-Object Interaction Reconstructioncited by 1
  3. 2026CLUTCH: Contextualized Language model for Unlocking Text-Conditioned Hand motion modelling in the wildcited by 1
  4. 2026UMo: Unified Sparse Motion Modeling for Real-Time Co-Speech Avatarscited by 1
  5. 2026KV-Control: Parameter-Efficient K/V Injection for Trajectory-Controlled Text-to-Motioncited by 1
  6. 2025PMG: Progressive Motion Generation via Sparse Anchor Postures Curriculum Learningcited by 0
  7. 2025AI-Generated Fall Data: Assessing LLMs and Diffusion Model for Wearable Fall Detectioncited by 0
  8. 2026Computer-Aided Design Generation by Cascaded Discrete Diffusion Modelcited by 0
  9. 2026EgoExoMoCap: Distributed Ego-Exo Human Motion Capturecited by 0
  10. 2026Capturing Uncertainty in Human Motion for Representation Learning in Soccercited by 0
  11. 2026Open-UniMo: Towards Unified Motion-Language Understanding and Generation in the Open Worldcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Diffusion Motion: Generate Text-Guided 3D Human Motion by Diffusion Model
  2. 2023AMD: Autoregressive Motion Diffusion
  3. 2025PackDiT: Joint Human Motion and Text Generation via Mutual Prompting
  4. 2026BiMoGen: Bidirectional Motion-Text Generation via Unified Masked Discrete Diffusion
  5. 2023DiffusionPhase: Motion Diffusion in Frequency Domain
  6. 2023Synthesizing Long-Term Human Motions with Diffusion Models via Coherent Sampling
  7. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression
  8. 2026DC-Motion: Decoupling Structure and Details via Discrete-Continuous Tokens for Human Motion Generation