# AnimateDiff-Lightning: Cross-Model Diffusion Distillation

arXiv: https://arxiv.org/abs/2403.12706 · HTML: https://mlsanity.com/paper/2403.12706/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 33 of its 45 references that are in the map.
- First posted on arXiv 19 Mar 2024.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 30 of 33)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://mlsanity.com/paper/2310.04378/) (2023)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [Structure and Content-Guided Video Synthesis with Diffusion Models](https://mlsanity.com/paper/2302.03011/) (2023)
- [MagicVideo: Efficient Video Generation With Latent Diffusion Models](https://mlsanity.com/paper/2211.11018/) (2022)
- [Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion](https://mlsanity.com/paper/2310.02279/) (2023)
- [Improved Techniques for Training Consistency Models](https://mlsanity.com/paper/2310.14189/) (2023)
- [InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation](https://mlsanity.com/paper/2309.06380/) (2023)
- [Effective Whole-body Pose Estimation with Two-stages Distillation](https://mlsanity.com/paper/2307.15880/) (2023)
- [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://mlsanity.com/paper/2305.08891/) (2023)

## Built on by (most cited first, 30 of 47)

- [From Slow Bidirectional to Fast Autoregressive Video Diffusion Models](https://mlsanity.com/paper/2412.07772/) (2024)
- [Diffusion Adversarial Post-Training for One-Step Video Generation](https://mlsanity.com/paper/2501.08316/) (2025)
- [PuLID: Pure and Lightning ID Customization via Contrastive Alignment](https://mlsanity.com/paper/2404.16022/) (2024)
- [FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality](https://mlsanity.com/paper/2410.19355/) (2024)
- [Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation](https://mlsanity.com/paper/2506.09350/) (2025)
- [SeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Training](https://mlsanity.com/paper/2506.05301/) (2025)
- [Distribution Matching Distillation Meets Reinforcement Learning](https://mlsanity.com/paper/2511.13649/) (2025)
- [SF-V: Single Forward Video Generation Model](https://mlsanity.com/paper/2406.04324/) (2024)
- [Motion Consistency Model: Accelerating Video Diffusion with Disentangled Motion-Appearance Distillation](https://mlsanity.com/paper/2406.06890/) (2024)
- [Looking Backward: Streaming Video-to-Video Translation with Feature Banks](https://mlsanity.com/paper/2405.15757/) (2024)
- [ProcessPainter: Learn Painting Process from Sequence Data](https://mlsanity.com/paper/2406.06062/) (2024)
- [SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Device](https://mlsanity.com/paper/2412.10494/) (2024)
- [Dual-Expert Consistency Model for Efficient and High-Quality Video Generation](https://mlsanity.com/paper/2506.03123/) (2025)
- [DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencoder](https://mlsanity.com/paper/2509.25182/) (2025)
- [A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generation](https://mlsanity.com/paper/2510.19755/) (2025)
- [On-device Sora: Enabling Training-Free Diffusion-based Text-to-Video Generation for Mobile Devices](https://mlsanity.com/paper/2502.04363/) (2025)
- [AccVideo: Accelerating Video Diffusion Model with Synthetic Dataset](https://mlsanity.com/paper/2503.19462/) (2025)
- [Your One-Stop Solution for AI-Generated Video Detection](https://mlsanity.com/paper/2601.11035/) (2026)
- [Adversarial Flow Models](https://mlsanity.com/paper/2511.22475/) (2025)
- [Accelerating Video Diffusion Models via Distribution Matching](https://mlsanity.com/paper/2412.05899/) (2024)
- [DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space](https://mlsanity.com/paper/2509.25180/) (2025)
- [Understanding Attention Mechanism in Video Diffusion Models](https://mlsanity.com/paper/2504.12027/) (2025)
- [LLIA -- Enabling Low-Latency Interactive Avatars: Real-Time Audio-Driven Portrait Video Generation with Diffusion Models](https://mlsanity.com/paper/2506.05806/) (2025)
- [Efficient Video Diffusion Models: Advancements and Challenges](https://mlsanity.com/paper/2604.15911/) (2026)
- [Protecting Your Video Content: Disrupting Automated Video-based LLM Annotations](https://mlsanity.com/paper/2503.21824/) (2025)
- [Continuous Adversarial Flow Models](https://mlsanity.com/paper/2604.11521/) (2026)
- [Human Action CLIPs: Detecting AI-generated Human Motion](https://mlsanity.com/paper/2412.00526/) (2024)
- [Inference-Time Diffusion Model Distillation](https://mlsanity.com/paper/2412.08871/) (2024)
- [DriftWorld: Fast World Modeling through Drifting](https://mlsanity.com/paper/2607.15065/) (2026)
- [A Large-scale Universal Evaluation Benchmark For Face Forgery Detection](https://mlsanity.com/paper/2406.09181/) (2024)

## Abstract (arXiv)

We present AnimateDiff-Lightning for lightning-fast video generation. Our model uses progressive adversarial diffusion distillation to achieve new state-of-the-art in few-step video generation. We discuss our modifications to adapt it for the video modality. Furthermore, we propose to simultaneously distill the probability flow of multiple base diffusion models, resulting in a single distilled motion module with broader style compatibility. We are pleased to release our distilled AnimateDiff-Lightning model for the community's use.