MotionRFT: Unified Reinforcement Fine-Tuning for Text-to-Motion Generation
Not on the map. Why?
Abstract (arXiv)
Text-to-motion generation has advanced with diffusion- and flow-based generative models, yet supervised pretraining remains insufficient to align models with high-level objectives such as semantic consistency, realism, and human preference. Existing post-training methods have key limitations: they (1) target a specific motion representation, such as joints, (2) optimize a particular aspect, such as text-motion alignment, and may compromise other factors; and (3) incur substantial computational overhead, data dependence, and coarse-grained optimization. We present a reinforcement fine-tuning framework that comprises a heterogeneous-representation, multi-dimensional reward model, MotionReward, and an efficient, fine-grained fine-tuning method, EasyTune. To obtain a unified semantics representation, MotionReward maps heterogeneous motions into a shared semantic space anchored by text, enabling multidimensional reward learning; Self-refinement Preference Learning further enhances semantics without additional annotations. For efficient and effective fine-tuning, we identify the recursive gradient dependence across denoising steps as the key bottleneck, and propose EasyTune, which optimizes step-wise rather than over the full trajectory, yielding dense, fine-grained, and memory-efficient updates. Extensive experiments validate the effectiveness of our framework, achieving FID 0.132 at 22.10 GB peak memory for MLD model and saving up to 15.22 GB over DRaFT. It reduces FID by 22.9% on joint-based ACMDM, and achieves a 12.6% R-Precision gain and 23.3% FID improvement on rotation-based HY Motion. Our project page with code is publicly available.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 62 of its 69 references that are in the map. First posted on arXiv 28 Mar 2026.
Builds on
62 of 69 references.
References · 62
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
- 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
- 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
- 2022Human Motion Diffusion Modelcited by 1,062
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
- 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
- 2023Training Diffusion Models with Reinforcement Learningcited by 939
- 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
- 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
38 more
- 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
- 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
- 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
- 2023MotionGPT: Human Motion as a Foreign Languagecited by 521
- 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 497
- 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
- 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
- 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
- 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 388
- 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
- 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 365
- 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
- 2017On human motion prediction using recurrent neural networkscited by 336
- 2016The KIT Motion-Language Datasetcited by 334
- 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
- 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 282
- 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
- 2023TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesiscited by 194
- 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 164
- 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 121
- 2023AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanismcited by 116
- 2024Motion Mamba: Efficient and Long Sequence Motion Generationcited by 111
- 2018QuaterNet: A Quaternion-based Recurrent Model for Human Motioncited by 108
- 2023Human Motion Generation: A Surveycited by 95
- 2024Golden Noise for Diffusion Models: A Learning Frameworkcited by 91
- 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 80
- 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 78
- 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 77
- 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 67
- 2024CoMo: Controllable Motion Generation through Language Guided Pose Code Editingcited by 66
- 2024Deep Reward Supervisions for Tuning Text-to-Image Diffusion Modelscited by 59
- 2024LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioningcited by 58
- 2023Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generationcited by 56
- 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practicescited by 39
- 2025HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generationcited by 32
- 2024DyMO: Training-Free Diffusion Model Alignment with Dynamic Multi-Objective Schedulingcited by 27
- 2025Absolute Coordinates Make Motion Generation Easycited by 27
50 most cited of 62 All 62 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Exploring Text-to-Motion Generation with Human Preference
- 2024MotionRL: Align Text-to-Motion Generation to Human Preferences with Multi-Reward Reinforcement Learning
- 2025No MoCap Needed: Post-Training Motion Diffusion Models with Reinforcement Learning using Only Textual Prompts
- 2025Motion-R1: Enhancing Motion Generation with Decomposed Chain-of-Thought and RL Binding
- 2025MotionFlux: Efficient Text-Guided Motion Generation through Rectified Flow Matching and Preference Alignment
- 2023Act As You Wish: Fine-Grained Control of Motion Diffusion Model with Hierarchical Semantic Graphs
- 2024MoDiPO: text-to-motion alignment via AI-feedback-driven Direct Preference Optimization
- 2026UniMo: Unified Motion Generation and Understanding with Chain of Thought