mlsanityData to 7 Oct 2026

ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model

arXiv 2304.01116 ↗First posted 3 Apr 2023cs.CVcited by 282

On the map: Retrieval-augmented generation → Multimodal RAG

also in 1 more
Abstract (arXiv)

3D human motion generation is crucial for creative industry. Recent advances rely on generative models with domain knowledge for text-driven motion generation, leading to substantial progress in capturing common motions. However, the performance on more diverse motions remains unsatisfactory. In this work, we propose ReMoDiffuse, a diffusion-model-based motion generation framework that integrates a retrieval mechanism to refine the denoising process. ReMoDiffuse enhances the generalizability and diversity of text-driven motion generation with three key designs: 1) Hybrid Retrieval finds appropriate references from the database in terms of both semantic and kinematic similarities. 2) Semantic-Modulated Transformer selectively absorbs retrieval knowledge, adapting to the difference between retrieved samples and the target motion sequence. 3) Condition Mixture better utilizes the retrieval database during inference, overcoming the scale sensitivity in classifier-free guidance. Extensive experiments demonstrate that ReMoDiffuse outperforms state-of-the-art methods by balancing both text-motion consistency and motion quality, especially for more diverse motion generation.

Key facts. Cited by 282 papers in the map (data through 7 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 26 of its 30 references that are in the map. First posted on arXiv 3 Apr 2023.

Builds on

26 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generator29 Sep

Other references · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  7. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  8. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  9. 2022Human Motion Diffusion Modelcited by 1,062
  10. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
  11. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
  12. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
13 more
  1. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 497
  2. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
  3. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 438
  4. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 406
  5. 2018Efficient Attention: Attention with Linear Complexitiescited by 366
  6. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
  7. 2016The KIT Motion-Language Datasetcited by 334
  8. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  9. 2021Synthesis of Compositional Animations from Textual Descriptionscited by 177
  10. 2022TEACH: Temporal Action Composition for 3D Humanscited by 173
  11. 2022KNN-Diffusion: Image Generation via Large-Scale Retrievalcited by 135
  12. 2021Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agentscited by 106
  13. 2022Balanced MSE for Imbalanced Visual Regressioncited by 89

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 31; 2024: 80; 2025: 95; 2026: 76.

282 papers.

Sort:

By area:

  1. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  2. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 388
  3. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  4. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  5. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 181
  6. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 164
  7. 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 154
  8. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 137
  9. 2023FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editingcited by 105
  10. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 94
  11. 2023Generative Image Dynamicscited by 93
  12. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 89
38 more
  1. 2023EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generationcited by 88
  2. 2023Diffusion Models for Reinforcement Learning: A Surveycited by 84
  3. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 78
  4. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 67
  5. 2024CoMo: Controllable Motion Generation through Language Guided Pose Code Editingcited by 66
  6. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  7. 2023AvatarGPT: All-in-One Framework for Motion Understanding, Planning, Generation and Beyondcited by 63
  8. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 63
  9. 2024LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioningcited by 58
  10. 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understandingcited by 58
  11. 2025VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documentscited by 57
  12. 2023Interactive Character Control with Auto-Regressive Motion Diffusion Modelscited by 52
  13. 2024ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Modelcited by 52
  14. 2024ParCo: Part-Coordinating Text-to-Motion Synthesiscited by 51
  15. 2024StableMoFusion: Towards Robust and Efficient Diffusion-based Motion Generation Frameworkcited by 48
  16. 2024RoHM: Robust Human Motion Reconstruction via Diffusioncited by 46
  17. 2024THOR: Text to Human-Object Interaction Diffusion via Relation Interventioncited by 46
  18. 2024Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompanimentcited by 46
  19. 2024The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motioncited by 46
  20. 2023Act As You Wish: Fine-Grained Control of Motion Diffusion Model with Hierarchical Semantic Graphscited by 44
  21. 2025SnapMoGen: Human Motion Generation from Expressive Textscited by 44
  22. 2023Digital Life Project: Autonomous 3D Characters with Social Intelligencecited by 43
  23. 2025GENMO: A GENeralist Model for Human MOtioncited by 43
  24. 2024FreeMotion: A Unified Framework for Number-free Text-to-Motion Synthesiscited by 42
  25. 2024MoGenTS: Motion Generation based on Spatial-Temporal Joint Modelingcited by 40
  26. 2025Motion Anything: Any to Motion Generationcited by 40
  27. 2025SALAD: Skeleton-aware Latent Diffusion for Text-driven Motion Generation and Editingcited by 37
  28. 2023Motion Flow Matching for Human Motion Synthesis and Editingcited by 36
  29. 2025MotionGPT3: Human Motion as a Second Modalitycited by 36
  30. 2024Retrieval-Augmented Diffusion Models for Time Series Forecastingcited by 35
  31. 2024LaserHuman: Language-guided Scene-aware Human Motion Generation in Free Environmentcited by 33
  32. 2025Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlookcited by 32
  33. 2025HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generationcited by 32
  34. 2024MotionChain: Conversational Motion Controllers via Multimodal Promptscited by 30
  35. 2024EnergyMoGen: Compositional Human Motion Generation with Energy-Based Diffusion Model in Latent Spacecited by 30
  36. 2024in2IN: Leveraging individual Information to Generate Human INteractionscited by 29
  37. 2024Unimotion: Unifying 3D Human Motion Synthesis and Understandingcited by 29
  38. 2024EgoLM: Multi-Modal Language Model of Egocentric Motionscited by 28

50 most cited of 282 All 282 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation
  2. 2022Diffusion Motion: Generate Text-Guided 3D Human Motion by Diffusion Model
  3. 2023Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model
  4. 2026From Diffusion to Flow: Efficient Motion Generation in MotionGPT3
  5. 2024RMD: A Simple Baseline for More General Human Motion Generation via Training-free Retrieval-Augmented Motion Diffuse
  6. 2025Back to Basics: Motion Representation Matters for Human Motion Generation Using Diffusion Model
  7. 2023AMD: Autoregressive Motion Diffusion
  8. 2023Priority-Centric Human Motion Generation in Discrete Latent Space