No MoCap Needed: Post-Training Motion Diffusion Models with Reinforcement Learning using Only Textual Prompts
On the map: Diffusion models → Video generation
also in 1 more
Abstract (arXiv)
Diffusion models have recently advanced human motion generation, producing realistic and diverse animations from textual prompts. However, adapting these models to unseen actions or styles typically requires additional motion capture data and full retraining, which is costly and difficult to scale. We propose a post-training framework based on Reinforcement Learning that fine-tunes pretrained motion diffusion models using only textual prompts, without requiring any motion ground truth. Our approach employs a pretrained text-motion retrieval network as a reward signal and optimizes the diffusion policy with Denoising Diffusion Policy Optimization, effectively shifting the model's generative distribution toward the target domain without relying on paired motion data. We evaluate our method on cross-dataset adaptation and leave-one-out motion experiments using the HumanML3D and KIT-ML datasets across both latent- and joint-space diffusion architectures. Results from quantitative metrics and user studies show that our approach consistently improves the quality and diversity of generated motions, while preserving performance on the original distribution. Our approach is a flexible, data-efficient, and privacy-preserving solution for motion adaptation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 32 of its 48 references that are in the map. First posted on arXiv 8 Oct 2025.
Builds on
32 of 48 references.
Closest ancestors · 6
Same direction, newest first
- 2024ReinDiffuse: Crafting Physically Plausible Motions with Reinforced Diffusion Model9 Oct
- 2024Generation of Complex 3D Human Motion by Temporal and Spatial Composition of Diffusion Models18 Sep
- 2024StableMoFusion: Towards Robust and Efficient Diffusion-based Motion Generation Framework9 May
- 2022Executing your Commands via Motion Diffusion in Latent Space8 Dec
- 2022FLAME: Free-form Language-based Motion Synthesis & Editing1 Sep
- 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model31 Aug
Other references · 26
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2023Training Diffusion Models with Reinforcement Learningcited by 982
- 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 938
- 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
- 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
- 2022TEMOS: Generating diverse human motions from textual descriptionscited by 492
- 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 485
- 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
- 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 448
14 more
- 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 400
- 2016The KIT Motion-Language Datasetcited by 338
- 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 163
- 2018Neural probabilistic motor primitives for humanoid controlcited by 97
- 2024Multi-Track Timeline Control for Text-Driven 3D Human Motion Generationcited by 91
- 2024Large-scale Reinforcement Learning for Diffusion Modelscited by 78
- 2022ActFormer: A GAN-based Transformer towards General Action-Conditioned 3D Human Motion Generationcited by 77
- 2018BiHMP-GAN: Bidirectional 3D Human Motion Prediction GANcited by 55
- 2024TextCraftor: Your Text Encoder Can be Image Quality Controllercited by 27
- 2024Morph: A Motion-free Physics Optimization Framework for Human Motion Generationcited by 13
- 2024Powerful and Flexible: Personalized Text-to-Image Generation via Reinforcement Learningcited by 11
- 2024MotionRL: Align Text-to-Motion Generation to Human Preferences with Multi-Reward Reinforcement Learningcited by 10
- 2021Human Motion Prediction Using Manifold-Aware Wasserstein GANcited by 8
- 2024Learning Generalizable Human Motion Generator with Reinforcement Learningcited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2023Human Motion Diffusion as a Generative Prior
- 2022Diffusion Motion: Generate Text-Guided 3D Human Motion by Diffusion Model
- 2026MotionRFT: Unified Reinforcement Fine-Tuning for Text-to-Motion Generation
- 2024Towards motion from video diffusion models
- 2026From Diffusion to Flow: Efficient Motion Generation in MotionGPT3
- 2023DiffusionPhase: Motion Diffusion in Frequency Domain
- 2026Reconstruction-Anchored Diffusion Model for Text-to-Motion Generation
- 2022Unifying Human Motion Synthesis and Style Transfer with Denoising Diffusion Probabilistic Models