mlsanityData to 7 Oct 2026

MotionRFT: Unified Reinforcement Fine-Tuning for Text-to-Motion Generation

arXiv 2603.27185 ↗First posted 28 Mar 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Text-to-motion generation has advanced with diffusion- and flow-based generative models, yet supervised pretraining remains insufficient to align models with high-level objectives such as semantic consistency, realism, and human preference. Existing post-training methods have key limitations: they (1) target a specific motion representation, such as joints, (2) optimize a particular aspect, such as text-motion alignment, and may compromise other factors; and (3) incur substantial computational overhead, data dependence, and coarse-grained optimization. We present a reinforcement fine-tuning framework that comprises a heterogeneous-representation, multi-dimensional reward model, MotionReward, and an efficient, fine-grained fine-tuning method, EasyTune. To obtain a unified semantics representation, MotionReward maps heterogeneous motions into a shared semantic space anchored by text, enabling multidimensional reward learning; Self-refinement Preference Learning further enhances semantics without additional annotations. For efficient and effective fine-tuning, we identify the recursive gradient dependence across denoising steps as the key bottleneck, and propose EasyTune, which optimizes step-wise rather than over the full trajectory, yielding dense, fine-grained, and memory-efficient updates. Extensive experiments validate the effectiveness of our framework, achieving FID 0.132 at 22.10 GB peak memory for MLD model and saving up to 15.22 GB over DRaFT. It reduces FID by 22.9% on joint-based ACMDM, and achieves a 12.6% R-Precision gain and 23.3% FID improvement on rotation-based HY Motion. Our project page with code is publicly available.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 62 of its 69 references that are in the map. First posted on arXiv 28 Mar 2026.

Builds on

62 of 69 references.

References · 62

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2022Flow Matching for Generative Modelingcited by 5,426
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  5. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  6. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  7. 2022Human Motion Diffusion Modelcited by 1,062
  8. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  9. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  10. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  11. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
  12. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
38 more
  1. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  2. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
  3. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
  4. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
  5. 2023MotionGPT: Human Motion as a Foreign Languagecited by 521
  6. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 497
  7. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
  8. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
  9. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
  10. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 388
  11. 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
  12. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 365
  13. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
  14. 2017On human motion prediction using recurrent neural networkscited by 336
  15. 2016The KIT Motion-Language Datasetcited by 334
  16. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
  17. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 282
  18. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
  19. 2023TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesiscited by 194
  20. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 164
  21. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 121
  22. 2023AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanismcited by 116
  23. 2024Motion Mamba: Efficient and Long Sequence Motion Generationcited by 111
  24. 2018QuaterNet: A Quaternion-based Recurrent Model for Human Motioncited by 108
  25. 2023Human Motion Generation: A Surveycited by 95
  26. 2024Golden Noise for Diffusion Models: A Learning Frameworkcited by 91
  27. 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 80
  28. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 78
  29. 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 77
  30. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 67
  31. 2024CoMo: Controllable Motion Generation through Language Guided Pose Code Editingcited by 66
  32. 2024Deep Reward Supervisions for Tuning Text-to-Image Diffusion Modelscited by 59
  33. 2024LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioningcited by 58
  34. 2023Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generationcited by 56
  35. 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practicescited by 39
  36. 2025HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generationcited by 32
  37. 2024DyMO: Training-Free Diffusion Model Alignment with Dynamic Multi-Objective Schedulingcited by 27
  38. 2025Absolute Coordinates Make Motion Generation Easycited by 27

50 most cited of 62 All 62 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Exploring Text-to-Motion Generation with Human Preference
  2. 2024MotionRL: Align Text-to-Motion Generation to Human Preferences with Multi-Reward Reinforcement Learning
  3. 2025No MoCap Needed: Post-Training Motion Diffusion Models with Reinforcement Learning using Only Textual Prompts
  4. 2025Motion-R1: Enhancing Motion Generation with Decomposed Chain-of-Thought and RL Binding
  5. 2025MotionFlux: Efficient Text-Guided Motion Generation through Rectified Flow Matching and Preference Alignment
  6. 2023Act As You Wish: Fine-Grained Control of Motion Diffusion Model with Hierarchical Semantic Graphs
  7. 2024MoDiPO: text-to-motion alignment via AI-feedback-driven Direct Preference Optimization
  8. 2026UniMo: Unified Motion Generation and Understanding with Chain of Thought