mlsanityData to 7 Oct 2026

VideoDPO: Omni-Preference Alignment for Video Diffusion Generation

arXiv 2412.14167 ↗First posted 18 Dec 2024cs.CV cs.AI cs.LGcited by 102

On the map: Diffusion models → Video generation

also in 1 more
Abstract (arXiv)

Recent progress in generative diffusion models has greatly advanced text-to-video generation. While text-to-video models trained on large-scale, diverse datasets can produce varied outputs, these generations often deviate from user preferences, highlighting the need for preference alignment on pre-trained models. Although Direct Preference Optimization (DPO) has demonstrated significant improvements in language and image generation, we pioneer its adaptation to video diffusion models and propose a VideoDPO pipeline by making several key adjustments. Unlike previous image alignment methods that focus solely on either (i) visual quality or (ii) semantic alignment between text and videos, we comprehensively consider both dimensions and construct a preference score accordingly, which we term the OmniScore. We design a pipeline to automatically collect preference pair data based on the proposed OmniScore and discover that re-weighting these pairs based on the score significantly impacts overall preference alignment. Our experiments demonstrate substantial improvements in both visual quality and semantic alignment, ensuring that no preference aspect is neglected. Code and data will be shared at https://videodpo.github.io/.

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 56 of its 65 references that are in the map. First posted on arXiv 18 Dec 2024.

Builds on

56 of 65 references.

Closest ancestors · 13

Same direction, newest first

  1. 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
  2. 2024T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Design8 Oct
  3. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  4. 2024Video Diffusion Alignment via Reward Gradients11 Jul
  5. 2024VEnhancer: Generative Space-Time Enhancement for Video Generation10 Jul
  6. 2024T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback29 May
  7. 2024VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Models10 Mar
  8. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models17 Jan
  9. 2023InstructVideo: Instructing Video Diffusion Models with Human Feedback19 Dec
  10. 2023ARTboldsymbol·V: Auto-Regressive Text-to-Video Generation with Diffusion Models30 Nov
  11. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  12. 2023ModelScope Text-to-Video Technical Report12 Aug
1 more
  1. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation23 Nov

Other references · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2016Improved Techniques for Training GANscited by 2,938
  10. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  11. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  12. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
31 more
  1. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  2. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  3. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  4. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  5. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  6. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
  7. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  8. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  9. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
  10. 2023SLiC-HF: Sequence Likelihood Calibration with Human Feedbackcited by 395
  11. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 390
  12. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  13. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 312
  14. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 278
  15. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpaintingcited by 229
  16. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
  17. 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 177
  18. 2023Holistic Evaluation of Text-To-Image Modelscited by 160
  19. 2023ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Modelscited by 127
  20. 2023FETV: A Benchmark for Fine-Grained Evaluation of Open-Domain Text-to-Video Generationcited by 119
  21. 2023Animate-A-Story: Storytelling with Retrieval-Augmented Video Generationcited by 89
  22. 2024Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimizationcited by 75
  23. 2024ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generationcited by 74
  24. 2024IGOR: Image-GOal Representations are the Atomic Control Units for Foundation Models in Embodied AIcited by 68
  25. 2024Evaluation of Text-to-Video Generation Models: A Dynamics Perspectivecited by 66
  26. 2024Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videoscited by 54
  27. 2024Follow-Your-Click: Open-domain Regional Image Animation via Short Promptscited by 52
  28. 2024Towards A Better Metric for Text-to-Video Generationcited by 50
  29. 2024Make a Cheap Scaling: A Self-Cascade Diffusion Model for Higher-Resolution Adaptationcited by 50
  30. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  31. 2023StoryBench: A Multifaceted Benchmark for Continuous Story Visualizationcited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 55; 2026: 46.

102 papers.

Sort:

By area:

  1. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
  2. 2025Improving Video Generation with Human Feedbackcited by 267
  3. 2025Unified Reward Model for Multimodal Understanding and Generationcited by 194
  4. 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuningcited by 86
  5. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 86
  6. 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillationcited by 75
  7. 2025BranchGRPO: Stable and Efficient GRPO with Structured Branching in Diffusion Modelscited by 59
  8. 2025Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Searchcited by 40
  9. 2024OnlineVPO: Align Video Diffusion Model with Online Video-Centric Preference Optimizationcited by 38
  10. 2025Scaling Image and Video Generation via Test-Time Evolutionary Searchcited by 38
  11. 2025DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Modelscited by 32
  12. 2025VQ-Insight: Teaching VLMs for AI-Generated Video Quality Understanding via Progressive Visual Reinforcement Learningcited by 26
38 more
  1. 2025PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generationcited by 22
  2. 2026JavisDiT++: Unified Modeling and Optimization for Joint Audio-Video Generationcited by 22
  3. 2025VideoUFO: A Million-Scale User-Focused Dataset for Text-to-Video Generationcited by 20
  4. 2025Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generationcited by 20
  5. 2026MultiWorld: Scalable Multi-Agent Multi-View Video World Modelscited by 19
  6. 2025MagicID: Hybrid Preference Optimization for ID-Consistent and Dynamic-Preserved Video Customizationcited by 17
  7. 2025Reasoning Physical Video Generation with Diffusion Timestep Tokens via Reinforcement Learningcited by 17
  8. 2025Hallo4: High-Fidelity Dynamic Portrait Animation via Direct Preference Optimizationcited by 16
  9. 2025VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Modelscited by 14
  10. 2025PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selectioncited by 14
  11. 2025Dual-IPO: Dual-Iterative Preference Optimization for Text-to-Video Generationcited by 12
  12. 2025ExeSQL: Self-Taught Text-to-SQL Models with Execution-Driven Bootstrapping for SQL Dialectscited by 11
  13. 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advancescited by 11
  14. 2025VR-Thinker: Boosting Video Reward Models through Thinking-with-Image Reasoningcited by 11
  15. 2025Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferencescited by 11
  16. 2026Unified Personalized Reward Model for Vision Generationcited by 11
  17. 2026UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generationcited by 11
  18. 2025PhysMaster: Mastering Physical Representation for Video Generation via Reinforcement Learningcited by 10
  19. 2025Identity-GRPO: Optimizing Multi-Human Identity-preserving Video Generation via Reinforcement Learningcited by 9
  20. 2025Fast Autoregressive Video Generation with Diagonal Decodingcited by 8
  21. 2025Aligning Anime Video Generation with Human Feedbackcited by 8
  22. 2026TAGRPO: Boosting GRPO on Image-to-Video Generation with Direct Trajectory Alignmentcited by 8
  23. 2025Towards Self-Improvement of Diffusion Models via Group Preference Optimizationcited by 7
  24. 2025AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animationcited by 7
  25. 2025Taming Camera-Controlled Video Generation with Verifiable Geometry Rewardcited by 7
  26. 2025PaCo-RL: Advancing Reinforcement Learning for Consistent Image Generation with Pairwise Reward Modelingcited by 7
  27. 2026VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine Systemcited by 7
  28. 2025RealDPO: Real or Not Real, that is the Preferencecited by 6
  29. 2025Discriminator-Free Direct Preference Optimization for Video Diffusioncited by 5
  30. 2025IC-World: In-Context Generation for Shared World Modelingcited by 5
  31. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 5
  32. 2026TeleBoost: A Systematic Alignment Framework for High-Fidelity, Controllable, and Robust Video Generationcited by 5
  33. 2026QualiTeacher: Quality-Conditioned Pseudo-Labeling for Real-World Image Restorationcited by 5
  34. 2026MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learningcited by 5
  35. 2025MultiCrafter: High-Fidelity Multi-Subject Generation via Disentangled Attention and Identity-Aware Preference Alignmentcited by 4
  36. 2025Learning What to Trust: Bayesian Prior-Guided Optimization for Visual Generationcited by 4
  37. 2026HP-Edit: A Human-Preference Post-Training Framework for Image Editingcited by 4
  38. 2025FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for Audio-Driven Portrait Animationcited by 3

50 most cited of 102 All 102 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025HuViDPO:Enhancing Video Generation through Direct Preference Optimization for Human-Centric Alignment
  2. 2025BalancedDPO: Adaptive Multi-Metric Alignment
  3. 2026Temporal Concentration from Rollout Errors: Implicit Preference Optimization for Text-to-Video Diffusion
  4. 2025Free Lunch Alignment of Text-to-Image Diffusion Models without Preference Image Pairs
  5. 2026Diffusion-APO: Trajectory-Aware Direct Preference Alignment for Video Diffusion Transformers
  6. 2025Diffusion-SDPO: Safeguarded Direct Preference Optimization for Diffusion Models
  7. 2024SafeSora: Towards Safety Alignment of Text2Video Generation via a Human Preference Dataset
  8. 2024DyMO: Training-Free Diffusion Model Alignment with Dynamic Multi-Objective Scheduling