mlsanityData to 14 Oct 2026

SoPo: Text-to-Motion Generation Using Semi-Online Preference Optimization

arXiv 2412.05095 ↗First posted 6 Dec 2024cs.CVcited by 15

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

Text-to-motion generation is essential for advancing the creative industry but often presents challenges in producing consistent, realistic motions. To address this, we focus on fine-tuning text-to-motion models to consistently favor high-quality, human-preferred motions, a critical yet largely unexplored problem. In this work, we theoretically investigate the DPO under both online and offline settings, and reveal their respective limitation: overfitting in offline DPO, and biased sampling in online DPO. Building on our theoretical insights, we introduce Semi-online Preference Optimization (SoPo), a DPO-based method for training text-to-motion models using "semi-online" data pair, consisting of unpreferred motion from online distribution and preferred motion in offline datasets. This method leverages both online and offline DPO, allowing each to compensate for the other's limitations. Extensive experiments demonstrate that SoPo outperforms other preference alignment methods, with an MM-Dist of 3.25% (vs e.g. 0.76% of MoDiPO) on the MLD model, 2.91% (vs e.g. 0.66% of MoDiPO) on MDM model, respectively. Additionally, the MLD model fine-tuned by our SoPo surpasses the SoTA model in terms of R-precision and MM Dist. Visualization results also show the efficacy of our SoPo in preference alignment. Project page: https://xiaofeng-tan.github.io/projects/SoPo/ .

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 48 of its 70 references that are in the map. First posted on arXiv 6 Dec 2024.

Builds on

48 of 70 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024MoDiPO: text-to-motion alignment via AI-feedback-driven Direct Preference Optimization6 May
  2. 2024Deep Reward Supervisions for Tuning Text-to-Image Diffusion Models1 May
  3. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model22 Nov
  4. 2023Diffusion Model Alignment Using Direct Preference Optimization21 Nov
  5. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewards29 Sep

Other references · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  4. 2022Human Motion Diffusion Modelcited by 1,080
  5. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  6. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  7. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  8. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  9. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  10. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 642
  11. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  12. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
31 more
  1. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  2. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 492
  3. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
  4. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  5. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 400
  6. 2023Human Motion Diffusion as a Generative Priorcited by 357
  7. 2016The KIT Motion-Language Datasetcited by 338
  8. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 288
  9. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 287
  10. 2024Direct Language Model Alignment from Online AI Feedbackcited by 260
  11. 2023TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesiscited by 198
  12. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 168
  13. 2023MMM: Generative Masked Motion Modelcited by 131
  14. 2024Motion Mamba: Efficient and Long Sequence Motion Generationcited by 112
  15. 2024Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimizationcited by 97
  16. 2023Human Motion Generation: A Surveycited by 96
  17. 2024Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordancecited by 80
  18. 2023Fg-T2M: Fine-Grained Text-Driven Human Motion Generation via Diffusion Modelcited by 79
  19. 2022UDE: A Unified Driving Engine for Human Motion Generationcited by 67
  20. 2023Priority-Centric Human Motion Generation in Discrete Latent Spacecited by 67
  21. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  22. 2017Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networkscited by 61
  23. 2024Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHFcited by 57
  24. 2023OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllerscited by 47
  25. 2022Being Comes from Not-being: Open-vocabulary Text-to-Motion Generation with Wordless Trainingcited by 45
  26. 2024FreeMotion: A Unified Framework for Number-free Text-to-Motion Synthesiscited by 42
  27. 2024Towards Efficient Exact Optimization of Language Model Alignmentcited by 37
  28. 2024Alignment of Diffusion Models: Fundamentals, Challenges, and Futurecited by 35
  29. 2024Tuning Timestep-Distilled Diffusion Model Using Pairwise Sample Optimizationcited by 26
  30. 2023DiffDance: Cascaded Human Motion Diffusion Model for Dance Generationcited by 23
  31. 2023Realistic Human Motion Generation with Cross-Diffusion Modelscited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 10.

15 papers.

Sort:

By area:

  1. 2025ReAlign: Text-to-Motion Generation via Step-Aware Reward-Guided Alignmentcited by 13
  2. 2025EvolvingGrasp: Evolutionary Grasp Generation via Efficient Preference Alignmentcited by 6
  3. 2026ConsistentRFT: Reducing Visual Hallucinations in Flow-based Reinforcement Fine-Tuningcited by 6
  4. 2026Temporal Consistency-Aware Text-to-Motion Generationcited by 5
  5. 2025FineXtrol: Controllable Motion Generation via Fine-Grained Textcited by 4
  6. 2026EasyTune: Efficient Step-Aware Fine-Tuning for Diffusion-Based Motion Generationcited by 4
  7. 2025PAMD: Plausibility-Aware Motion Diffusion Model for Long Dance Generationcited by 3
  8. 2026Attribution as Retrieval: Model-Agnostic AI-Generated Image Attributioncited by 3
  9. 2026MotionRFT: Unified Reinforcement Fine-Tuning for Text-to-Motion Generationcited by 1
  10. 2026Coordinate-Based Dual-Constrained Autoregressive Motion Generationcited by 1
  11. 2025ReAlign: Bilingual Text-to-Motion Generation via Step-Aware Reward-Guided Alignmentcited by 0
  12. 2026Towards Highly-Constrained Human Motion Generation with Retrieval-Guided Diffusion Noise Optimizationcited by 0
3 more
  1. 2026Boosting Reinforcement Learning with Verifiable Rewards via Randomly Selected Few-Shot Guidancecited by 0
  2. 2026Training-free Controllable Human Motion Generation under Heterogeneous Constraintscited by 0
  3. 2026Getting Space Right: Agentic Rewards for Image Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Exploring Text-to-Motion Generation with Human Preference
  2. 2025RealDPO: Real or Not Real, that is the Preference
  3. 2025HuViDPO:Enhancing Video Generation through Direct Preference Optimization for Human-Centric Alignment
  4. 2025MotionFlux: Efficient Text-Guided Motion Generation through Rectified Flow Matching and Preference Alignment
  5. 2024Learning Generalizable Human Motion Generator with Reinforcement Learning
  6. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generation
  7. 2025DanceGRPO: Unleashing GRPO on Visual Generation
  8. 2024MotionRL: Align Text-to-Motion Generation to Human Preferences with Multi-Reward Reinforcement Learning