mlsanityData to 14 Oct 2026

Reward Lightning: Fast Video Generation via Homologous Preference Distillation

arXiv 2607.03960 ↗First posted 4 Jul 2026cs.CVcited by 0

On the map: Diffusion models → Video generation

also in 1 more
Abstract (arXiv)

Achieving simultaneous preference alignment and distillation acceleration in video diffusion models remains an open challenge. Existing methods optimize the two objectives over mismatched representation spaces, where improving one objective often compromises the other. To overcome this, we propose Reward Lightning, a unified framework that aligns and accelerates a video diffusion model within a single shared representation. Its central principle is homology: both objectives are evaluated on identical latent features, which mitigates the gradient conflicts that arise when they are optimized over disjoint representations. As a foundational component, we first introduce a latent reward model (LRM) that scores videos directly in the latent space, without decoding back to the pixel space. Building on the LRM, homologous preference distillation (HPD) reuses this shared backbone to perform adversarial distillation and preference alignment jointly, yielding few-step generators that remain faithful and well aligned. Extensive experiments demonstrate that the LRM surpasses pixel-level and latent-level reward baselines by 11.0% and 14.7% in preference accuracy, and that Reward Lightning generates high-fidelity videos in merely 1 to 4 steps, improving the average VBench score by 2.1% while leading in text alignment, motion quality, and visual quality. Project page: https://reward-lightning.github.io.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 56 of its 62 references that are in the map. First posted on arXiv 4 Jul 2026.

Builds on

56 of 62 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025TurboDiffusion: Accelerating Video Diffusion Models by 100-200 Times18 Dec
  2. 2025CascadeV: An Implementation of Wurstchen Architecture for Video Generation28 Jan
  3. 2025Improving Video Generation with Human Feedback23 Jan
  4. 2025Diffusion Adversarial Post-Training for One-Step Video Generation14 Jan
  5. 2024DOLLAR: Few-Step Video Generation via Distillation and Latent Reward Optimization20 Dec
  6. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  7. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  8. 2022Video Diffusion Models7 Apr

Other references · 48

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2022Flow Matching for Generative Modelingcited by 5,702
  8. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  10. 2017Deep reinforcement learning from human preferencescited by 3,912
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  12. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
36 more
  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  2. 2023Consistency Modelscited by 1,822
  3. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  4. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  5. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  6. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  7. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  8. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  9. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  10. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
  11. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
  12. 2023Adversarial Diffusion Distillationcited by 759
  13. 2022On Distillation of Guided Diffusion Modelscited by 725
  14. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 684
  15. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  16. 2025Mean Flows for One-step Generative Modelingcited by 561
  17. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusioncited by 459
  18. 2023Aligning Text-to-Image Models using Human Feedbackcited by 422
  19. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 287
  20. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 279
  21. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 266
  22. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 252
  23. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 245
  24. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
  25. 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generationcited by 156
  26. 2024Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Contentcited by 135
  27. 2025Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistencycited by 86
  28. 2024Phased Consistency Modelscited by 85
  29. 2024GenAI Arena: An Open Evaluation Platform for Generative Modelscited by 82
  30. 2024LiFT: Leveraging Human Feedback for Text-to-Video Model Alignmentcited by 53
  31. 2025Distribution Matching Distillation Meets Reinforcement Learningcited by 50
  32. 2025Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Modelscited by 35
  33. 2025Flash-DMD: Towards High-Fidelity Few-Step Image Generation with Efficient Distillation and Joint Reinforcement Learningcited by 17
  34. 2025Video Generation Models Are Good Latent Reward Modelscited by 16
  35. 2024UniFL: Improve Latent Diffusion Model via Unified Feedback Learningcited by 9
  36. 2025ContentV: Efficient Training of Video Generation Models with Limited Computecited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CoRe: Co-Evolving Reward Models for Mitigating Latent Reward Hacking in Video Diffusion Models
  2. 2024Video Diffusion Alignment via Reward Gradients
  3. 2026Joint Alignment and Distillation for Video Generation via Sample-Guided Distribution Matching
  4. 2026Through the PRISM: Preference Representation in Intermediate States of Video Diffusion Models
  5. 2026Aligning Human Sense: Calibrated Distributional Reward Learning for Video Generation
  6. 2026IsoVRBench: Video Reward Modeling Without Human Feedback
  7. 2025SoliReward: Mitigating Susceptibility to Reward Hacking and Annotation Noise in Video Generation Reward Models
  8. 2026GT-SVJ: Generative-Transformer-Based Self-Supervised Video Judge For Efficient Video Reward Modeling