mlsanityData to 14 Oct 2026

RealDPO: Real or Not Real, that is the Preference

arXiv 2510.14955 ↗First posted 16 Oct 2025cs.CV cs.AIcited by 6

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

Video generative models have recently achieved notable advancements in synthesis quality. However, generating complex motions remains a critical challenge, as existing models often struggle to produce natural, smooth, and contextually consistent movements. This gap between generated and real-world motions limits their practical applicability. To address this issue, we introduce RealDPO, a novel alignment paradigm that leverages real-world data as positive samples for preference learning, enabling more accurate motion synthesis. Unlike traditional supervised fine-tuning (SFT), which offers limited corrective feedback, RealDPO employs Direct Preference Optimization (DPO) with a tailored loss function to enhance motion realism. By contrasting real-world videos with erroneous model outputs, RealDPO enables iterative self-correction, progressively refining motion quality. To support post-training in complex motion synthesis, we propose RealAction-5K, a curated dataset of high-quality videos capturing human daily activities with rich and precise motion details. Extensive experiments demonstrate that RealDPO significantly improves video quality, text alignment, and motion realism compared to state-of-the-art models and existing preference optimization techniques.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 34 of its 38 references that are in the map. First posted on arXiv 16 Oct 2025.

Builds on

34 of 38 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generation14 Aug
  2. 2025DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Models4 Jun
  3. 2025Cycle Consistency as Reward: Learning Image-Text Alignment without Human Preferences2 Jun
  4. 2025Self-NPO: Data-Free Diffusion Model Enhancement via Truncated Diffusion Fine-Tuning17 May
  5. 2025VPO: Aligning Text-to-Video Generation Models with Prompt Optimization26 Mar
  6. 2025Unified Reward Model for Multimodal Understanding and Generation7 Mar
  7. 2025Improving Video Generation with Human Feedback23 Jan
  8. 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation30 Dec
  9. 2024OnlineVPO: Align Video Diffusion Model with Online Video-Centric Preference Optimization19 Dec
  10. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generation18 Dec
  11. 2024LiFT: Leveraging Human Feedback for Text-to-Video Model Alignment6 Dec
  12. 2023InstructVideo: Instructing Video Diffusion Models with Human Feedback19 Dec
5 more
  1. 2023Rich Human Feedback for Text-to-Image Generation15 Dec
  2. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model22 Nov
  3. 2023Diffusion Model Alignment Using Direct Preference Optimization21 Nov
  4. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewards29 Sep
  5. 2023Aligning Text-to-Image Models using Human Feedback23 Feb

Other references · 17

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  4. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  5. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  6. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  7. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  8. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  9. 2023ModelScope Text-to-Video Technical Reportcited by 658
  10. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  11. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 579
  12. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 471
5 more
  1. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 350
  2. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 298
  3. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
  4. 2025Implicit Reward as the Bridge: A Unified View of SFT and DPO Connectionscited by 17
  5. 2025SwiftVideo: A Unified Framework for Few-Step Video Generation through Trajectory-Distribution Alignmentcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024SoPo: Text-to-Motion Generation Using Semi-Online Preference Optimization
  2. 2026Mind the Generative Details: Direct Localized Detail Preference Optimization for Video Diffusion Models
  3. 2024MoDiPO: text-to-motion alignment via AI-feedback-driven Direct Preference Optimization
  4. 2025AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animation
  5. 2025Discriminator-Free Direct Preference Optimization for Video Diffusion
  6. 2026DynaVid: Learning to Generate Highly Dynamic Videos using Synthetic Motion Data
  7. 2026PhyCo: Learning Controllable Physical Priors for Generative Motion
  8. 2026Diffusion-APO: Trajectory-Aware Direct Preference Alignment for Video Diffusion Transformers