mlsanityData to 7 Oct 2026

ReinDiffuse: Crafting Physically Plausible Motions with Reinforced Diffusion Model

arXiv 2410.07296 ↗First posted 9 Oct 2024cs.CVcited by 25

On the map: Diffusion models → Video generation

Abstract (arXiv)

Generating human motion from textual descriptions is a challenging task. Existing methods either struggle with physical credibility or are limited by the complexities of physics simulations. In this paper, we present ReinDiffuse that combines reinforcement learning with motion diffusion model to generate physically credible human motions that align with textual descriptions. Our method adapts Motion Diffusion Model to output a parameterized distribution of actions, making them compatible with reinforcement learning paradigms. We employ reinforcement learning with the objective of maximizing physically plausible rewards to optimize motion generation for physical fidelity. Our approach outperforms existing state-of-the-art models on two major datasets, HumanML3D and KIT-ML, achieving significant improvements in physical plausibility and motion quality. Project: https://reindiffuse.github.io/

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 38 of its 52 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

38 of 52 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023InstructVideo: Instructing Video Diffusion Models with Human Feedback19 Dec
  2. 2023HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models with Minimal Feedback19 Dec
  3. 2023Guided Motion Diffusion for Controllable Human Motion Synthesis21 May
  4. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model3 Apr
  5. 2022Executing your Commands via Motion Diffusion in Latent Space8 Dec
  6. 2022PhysDiff: Physics-Guided Human Motion Diffusion Model5 Dec
  7. 2022Human Motion Diffusion Model29 Sep
  8. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model31 Aug

Other references · 30

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2017Neural Discrete Representation Learningcited by 4,650
  4. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  5. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  6. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  7. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  8. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
  9. 2023MotionGPT: Human Motion as a Foreign Languagecited by 521
  10. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
  11. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 479
  12. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
18 more
  1. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  2. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
  3. 2016The KIT Motion-Language Datasetcited by 334
  4. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
  5. 2022EDGE: Editable Dance Generation From Musiccited by 284
  6. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 233
  7. 2023Object Motion Guided Human Motion Synthesiscited by 206
  8. 2022Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Modelscited by 188
  9. 2022TEACH: Temporal Action Composition for 3D Humanscited by 173
  10. 2020Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenescited by 135
  11. 2021DanceFormer: Music Conditioned 3D Dance Generation with Parametric Motion Transformercited by 111
  12. 2023Synthesizing Physical Character-Scene Interactionscited by 102
  13. 2023Synthesizing Physically Plausible Human Motions in 3D Scenescited by 47
  14. 2019Learning to Sit: Synthesizing Human-Chair Interactions via Hierarchical Controlcited by 39
  15. 2020Perpetual Motion: Generating Unbounded Human Motioncited by 36
  16. 2023Tuning computer vision models with task rewardscited by 36
  17. 2023DiffMotion: Speech-Driven Gesture Synthesis Using Denoising Diffusion Modelcited by 22
  18. 2022Learning Variational Motion Prior for Video-based Motion Capturecited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 13; 2026: 9.

25 papers.

Sort:

By area:

  1. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 94
  2. 2025Diffuse-CLoC: Guided Diffusion for Physics-based Character Look-ahead Controlcited by 27
  3. 2025RL from Physical Feedback: Aligning Large Motion Models with Humanoid Controlcited by 10
  4. 2025ReAlign: Text-to-Motion Generation via Step-Aware Reward-Guided Alignmentcited by 10
  5. 2025ARFlow: Human Action-Reaction Flow Matching with Physical Guidancecited by 7
  6. 2026SafeMo: Linguistically Grounded Unlearning for Trustworthy Text-to-Motion Generationcited by 6
  7. 2024Conditional GAN for Enhancing Diffusion Models in Efficient and Authentic Global Gesture Generation from Audioscited by 5
  8. 2025PP-Motion: Physical-Perceptual Fidelity Evaluation for Human Motion Generationcited by 4
  9. 2025Uni-Inter: Unifying 3D Human Motion Synthesis Across Diverse Interaction Contextscited by 4
  10. 2026EasyTune: Efficient Step-Aware Fine-Tuning for Diffusion-Based Motion Generationcited by 3
  11. 2024ExpGest: Expressive Speaker Generation Using Diffusion Model and Hybrid Audio-Text Guidancecited by 2
  12. 2024RopeTP: Global Human Motion Recovery via Integrating Robust Pose Estimation with Diffusion Trajectory Priorcited by 2
13 more
  1. 2025Biomechanics-Guided Residual Approach to Generalizable Human Motion Generation and Estimationcited by 1
  2. 2025SimDiff: Simulator-constrained Diffusion Model for Physically Plausible Motion Generationcited by 1
  3. 2025No MoCap Needed: Post-Training Motion Diffusion Models with Reinforcement Learning using Only Textual Promptscited by 1
  4. 2026PhysMoDPO: Physically-Plausible Humanoid Motion with Preference Optimizationcited by 1
  5. 2025From 2D Alignment to 3D Plausibility: Unifying Heterogeneous 2D Priors and Penetration-Free Diffusion for Occlusion-Robust Two-Hand Reconstructioncited by 0
  6. 2025ReAlign: Bilingual Text-to-Motion Generation via Step-Aware Reward-Guided Alignmentcited by 0
  7. 2025Inter-Diffusion Generation Model of Speakers and Listeners for Effective Communicationcited by 0
  8. 2026Skeleton2Stage: Reward-Guided Fine-Tuning for Physically Plausible Dance Generationcited by 0
  9. 2026A Self-Supervised Approach on Motion Calibration for Enhancing Physical Plausibility in Text-to-Motioncited by 0
  10. 2026MotionRFT: Unified Reinforcement Fine-Tuning for Text-to-Motion Generationcited by 0
  11. 2026Re2MoGen: Open-Vocabulary Motion Generation via LLM Reasoning and Physics-Aware Refinementcited by 0
  12. 2026MotionGRPO: Overcoming Low Intra-Group Diversity in GRPO-Based Egocentric Motion Recoverycited by 0
  13. 2026Training-free Controllable Human Motion Generation under Heterogeneous Constraintscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024PDP: Physics-Based Character Animation via Diffusion Policy
  2. 2026ReActor: Reinforcement Learning for Physics-Aware Motion Retargeting
  3. 2025DemoDiffusion: One-Shot Human Imitation using pre-trained Diffusion Policy
  4. 2024Generation of Complex 3D Human Motion by Temporal and Spatial Composition of Diffusion Models
  5. 2023Human Motion Diffusion as a Generative Prior
  6. 2024CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control
  7. 2023AMD: Autoregressive Motion Diffusion
  8. 2024Towards motion from video diffusion models