Reward Lightning: Fast Video Generation via Homologous Preference Distillation
On the map: Diffusion models → Video generation
also in 1 more
Abstract (arXiv)
Achieving simultaneous preference alignment and distillation acceleration in video diffusion models remains an open challenge. Existing methods optimize the two objectives over mismatched representation spaces, where improving one objective often compromises the other. To overcome this, we propose Reward Lightning, a unified framework that aligns and accelerates a video diffusion model within a single shared representation. Its central principle is homology: both objectives are evaluated on identical latent features, which mitigates the gradient conflicts that arise when they are optimized over disjoint representations. As a foundational component, we first introduce a latent reward model (LRM) that scores videos directly in the latent space, without decoding back to the pixel space. Building on the LRM, homologous preference distillation (HPD) reuses this shared backbone to perform adversarial distillation and preference alignment jointly, yielding few-step generators that remain faithful and well aligned. Extensive experiments demonstrate that the LRM surpasses pixel-level and latent-level reward baselines by 11.0% and 14.7% in preference accuracy, and that Reward Lightning generates high-fidelity videos in merely 1 to 4 steps, improving the average VBench score by 2.1% while leading in text alignment, motion quality, and visual quality. Project page: https://reward-lightning.github.io.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 56 of its 62 references that are in the map. First posted on arXiv 4 Jul 2026.
Builds on
56 of 62 references.
Closest ancestors · 8
Same direction, newest first
- 2025TurboDiffusion: Accelerating Video Diffusion Models by 100-200 Times18 Dec
- 2025CascadeV: An Implementation of Wurstchen Architecture for Video Generation28 Jan
- 2025Improving Video Generation with Human Feedback23 Jan
- 2025Diffusion Adversarial Post-Training for One-Step Video Generation14 Jan
- 2024DOLLAR: Few-Step Video Generation via Distillation and Latent Reward Optimization20 Dec
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
- 2022Video Diffusion Models7 Apr
Other references · 48
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
36 more
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2023Consistency Modelscited by 1,822
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
- 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
- 2023Training Diffusion Models with Reinforcement Learningcited by 982
- 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
- 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
- 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
- 2023Adversarial Diffusion Distillationcited by 759
- 2022On Distillation of Guided Diffusion Modelscited by 725
- 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 684
- 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
- 2025Mean Flows for One-step Generative Modelingcited by 561
- 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusioncited by 459
- 2023Aligning Text-to-Image Models using Human Feedbackcited by 422
- 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 287
- 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 279
- 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 266
- 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 252
- 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 245
- 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
- 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generationcited by 156
- 2024Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Contentcited by 135
- 2025Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistencycited by 86
- 2024Phased Consistency Modelscited by 85
- 2024GenAI Arena: An Open Evaluation Platform for Generative Modelscited by 82
- 2024LiFT: Leveraging Human Feedback for Text-to-Video Model Alignmentcited by 53
- 2025Distribution Matching Distillation Meets Reinforcement Learningcited by 50
- 2025Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Modelscited by 35
- 2025Flash-DMD: Towards High-Fidelity Few-Step Image Generation with Efficient Distillation and Joint Reinforcement Learningcited by 17
- 2025Video Generation Models Are Good Latent Reward Modelscited by 16
- 2024UniFL: Improve Latent Diffusion Model via Unified Feedback Learningcited by 9
- 2025ContentV: Efficient Training of Video Generation Models with Limited Computecited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026CoRe: Co-Evolving Reward Models for Mitigating Latent Reward Hacking in Video Diffusion Models
- 2024Video Diffusion Alignment via Reward Gradients
- 2026Joint Alignment and Distillation for Video Generation via Sample-Guided Distribution Matching
- 2026Through the PRISM: Preference Representation in Intermediate States of Video Diffusion Models
- 2026Aligning Human Sense: Calibrated Distributional Reward Learning for Video Generation
- 2026IsoVRBench: Video Reward Modeling Without Human Feedback
- 2025SoliReward: Mitigating Susceptibility to Reward Hacking and Annotation Noise in Video Generation Reward Models
- 2026GT-SVJ: Generative-Transformer-Based Self-Supervised Video Judge For Efficient Video Reward Modeling