mlsanityData to 14 Oct 2026

InstructVideo: Instructing Video Diffusion Models with Human Feedback

arXiv 2312.12490 ↗First posted 19 Dec 2023cs.CV cs.AI cs.LGcited by 86

On the map: Diffusion models → Video generation

also in 1 more
Abstract (arXiv)

Diffusion models have emerged as the de facto paradigm for video generation. However, their reliance on web-scale data of varied quality often yields results that are visually unappealing and misaligned with the textual prompts. To tackle this problem, we propose InstructVideo to instruct text-to-video diffusion models with human feedback by reward fine-tuning. InstructVideo has two key ingredients: 1) To ameliorate the cost of reward fine-tuning induced by generating through the full DDIM sampling chain, we recast reward fine-tuning as editing. By leveraging the diffusion process to corrupt a sampled video, InstructVideo requires only partial inference of the DDIM sampling chain, reducing fine-tuning cost while improving fine-tuning efficiency. 2) To mitigate the absence of a dedicated video reward model for human preferences, we repurpose established image reward models, e.g., HPSv2. To this end, we propose Segmental Video Reward, a mechanism to provide reward signals based on segmental sparse sampling, and Temporally Attenuated Reward, a method that mitigates temporal modeling degradation during fine-tuning. Extensive experiments, both qualitative and quantitative, validate the practicality and efficacy of using image reward models in InstructVideo, significantly enhancing the visual quality of generated videos without compromising generalization capabilities. Code and models will be made publicly available.

Key facts. Cited by 86 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 97 of its 104 references that are in the map. First posted on arXiv 19 Dec 2023.

Builds on

97 of 104 references.

Closest ancestors · 29

Same direction, newest first

  1. 2023VideoLCM: Video Latent Consistency Model14 Dec
  2. 2023Hierarchical Spatio-temporal Decoupling for Text-to-Video Generation7 Dec
  3. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motion7 Dec
  4. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models7 Nov
  5. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction31 Oct
  6. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors18 Oct
  7. 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Models12 Oct
  8. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation27 Sep
  9. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models26 Sep
  10. 2023Reuse and Diffuse: Iterative Denoising for Text-to-Video Generation7 Sep
  11. 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editing18 Aug
  12. 2023Dual-Stream Diffusion Net for Text-to-Video Generation16 Aug
17 more
  1. 2023ModelScope Text-to-Video Technical Report12 Aug
  2. 2023GD-VDM: Generated Depth for better Diffusion-based Video Generation19 Jun
  3. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models17 May
  4. 2023Motion-Conditioned Diffusion Model for Controllable Video Synthesis27 Apr
  5. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  6. 2023Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generation17 Apr
  7. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators23 Mar
  8. 2023Pix2Video: Video Editing using Image Diffusion22 Mar
  9. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editing16 Mar
  10. 2023Video Probabilistic Diffusion Models in Projected Latent Space15 Feb
  11. 2023Structure and Content-Guided Video Synthesis with Diffusion Models6 Feb
  12. 2023Dreamix: Video Diffusion Models are General Video Editors2 Feb
  13. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation22 Dec
  14. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation23 Nov
  15. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Models20 Nov
  16. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct
  17. 2022Video Diffusion Models7 Apr

Other references · 68

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2022Training language models to follow instructions with human feedbackcited by 15,971
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  8. 2020Denoising Diffusion Implicit Modelscited by 9,781
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  10. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  11. 2022Classifier-Free Diffusion Guidancecited by 5,847
  12. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
38 more
  1. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  2. 2017Deep reinforcement learning from human preferencescited by 3,912
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  4. 2020Learning to summarize from human feedbackcited by 2,588
  5. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  7. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  8. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  9. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,476
  10. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  11. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  12. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  13. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  14. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  15. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  16. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  17. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  18. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  19. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  20. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  21. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 647
  22. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  23. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  24. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
  25. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 448
  26. 2018Scalable agent alignment via reward modeling: a research directioncited by 447
  27. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 442
  28. 2023Aligning Text-to-Image Models using Human Feedbackcited by 422
  29. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 314
  30. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 314
  31. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 301
  32. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 288
  33. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 287
  34. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 264
  35. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 256
  36. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  37. 2022MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generationcited by 227
  38. 2023Any-to-Any Generation via Composable Diffusioncited by 204

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 25; 2025: 41; 2026: 19.

86 papers.

Sort:

By area:

  1. 2025Improving Video Generation with Human Feedbackcited by 284
  2. 2025WorldModelBench: Judging Video Generation Models As World Modelscited by 124
  3. 2024Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advancescited by 110
  4. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 108
  5. 2024Video Diffusion Alignment via Reward Gradientscited by 107
  6. 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animationcited by 102
  7. 2024Animate-X: Universal Character Image Animation with Enhanced Motion Representationcited by 88
  8. 2024T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedbackcited by 87
  9. 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillationcited by 86
  10. 2024EasyAnimate: High-Performance Video Generation Framework with Hybrid Windows Attention and Reward Backpropagationcited by 77
  11. 2024T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Designcited by 54
  12. 2024Sora as a World Model? A Complete Survey on Text-to-Video Generationcited by 53
38 more
  1. 2024LiFT: Leveraging Human Feedback for Text-to-Video Model Alignmentcited by 53
  2. 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53
  3. 2023A Recipe for Scaling up Text-to-Video Generation with Text-free Videoscited by 51
  4. 2024Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedbackcited by 47
  5. 2024DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Controlcited by 43
  6. 2024OnlineVPO: Align Video Diffusion Model with Online Video-Centric Preference Optimizationcited by 41
  7. 2024Motion Consistency Model: Accelerating Video Diffusion with Disentangled Motion-Appearance Distillationcited by 39
  8. 2024Make Continual Learning Stronger via C-Flatcited by 35
  9. 2024Alignment of Diffusion Models: Fundamentals, Challenges, and Futurecited by 35
  10. 2025DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Modelscited by 34
  11. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  12. 2024FreeScale: Unleashing the Resolution of Diffusion Models via Tuning-Free Scale Fusioncited by 30
  13. 2024UniMLVG: Unified Framework for Multi-view Long Video Generation with Comprehensive Control Capabilities for Autonomous Drivingcited by 27
  14. 2024ReinDiffuse: Crafting Physically Plausible Motions with Reinforced Diffusion Modelcited by 26
  15. 2024StableV2V: Stablizing Shape Consistency in Video-to-Video Editingcited by 26
  16. 2025FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarioscited by 25
  17. 2025DreamRelation: Relation-Centric Video Customizationcited by 23
  18. 2025Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspectivecited by 22
  19. 2025Error Analyses of Auto-Regressive Video Diffusion Models: A Unified Frameworkcited by 20
  20. 2025InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignmentcited by 19
  21. 2024Aligning Human Motion Generation with Human Perceptionscited by 16
  22. 2025MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generationcited by 16
  23. 2025Video Generation Models Are Good Latent Reward Modelscited by 16
  24. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  25. 2025Dual-IPO: Dual-Iterative Preference Optimization for Text-to-Video Generationcited by 13
  26. 2025Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferencescited by 13
  27. 2026Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directionscited by 13
  28. 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advancescited by 12
  29. 2025UniLumos: Fast and Unified Image and Video Relighting with Physics-Plausible Feedbackcited by 12
  30. 2026Diffusion-DRF: Free, Rich, and Differentiable Reward for Video Diffusion Fine-Tuningcited by 10
  31. 2025ZeroFlow: Overcoming Catastrophic Forgetting is Easier than You Thinkcited by 9
  32. 2025VidSketch: Hand-drawn Sketch-Driven Video Generation with Diffusion Controlcited by 9
  33. 2025Aligning Anime Video Generation with Human Feedbackcited by 9
  34. 2025RDPO: Real Data Preference Optimization for Physics Consistency Video Generationcited by 9
  35. 2025Reinforcement Learning for Large Model: A Surveycited by 9
  36. 2025ImagerySearch: Adaptive Test-Time Search for Video Generation Beyond Semantic Dependency Constraintscited by 9
  37. 2024FlipSketch: Flipping Static Drawings to Text-Guided Sketch Animationscited by 8
  38. 2025Towards Physical Understanding in Video Generation: A 3D Point Regularization Approachcited by 8

50 most cited of 86 All 86 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Temporal Concentration from Rollout Errors: Implicit Preference Optimization for Text-to-Video Diffusion
  2. 2026A Systematic Post-Train Framework for Video Generation
  3. 2025Temporal In-Context Fine-Tuning with Temporal Reasoning for Versatile Control of Video Diffusion Models
  4. 2026CoRe: Co-Evolving Reward Models for Mitigating Latent Reward Hacking in Video Diffusion Models
  5. 2025GigaVideo-1: Advancing Video Generation via Automatic Feedback with 4 GPU-Hours Fine-Tuning
  6. 2023Control-A-Video: Controllable Text-to-Video Diffusion Models with Motion Prior and Reward Feedback Learning
  7. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
  8. 2026IsoVRBench: Video Reward Modeling Without Human Feedback