# MagicDistillation: Weak-to-Strong Video Distillation for Large-Scale Few-Step Synthesis

arXiv: https://arxiv.org/abs/2503.13319 · HTML: https://mlsanity.com/paper/2503.13319/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 37 of its 45 references that are in the map.
- First posted on arXiv 17 Mar 2025.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 30 of 37)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Improved Distribution Matching Distillation for Fast Image Synthesis](https://mlsanity.com/paper/2405.14867/) (2024)
- [ModelScope Text-to-Video Technical Report](https://mlsanity.com/paper/2308.06571/) (2023)
- [FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision](https://mlsanity.com/paper/2407.08608/) (2024)
- [SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation](https://mlsanity.com/paper/2211.12194/) (2022)
- [I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models](https://mlsanity.com/paper/2311.04145/) (2023)
- [ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning](https://mlsanity.com/paper/2104.07857/) (2021)
- [CelebV-HQ: A Large-Scale Video Facial Attributes Dataset](https://mlsanity.com/paper/2207.12393/) (2022)
- [SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction](https://mlsanity.com/paper/2310.20700/) (2023)
- [T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation](https://mlsanity.com/paper/2407.14505/) (2024)
- [VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolution](https://mlsanity.com/paper/2205.03409/) (2022)
- [Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content](https://mlsanity.com/paper/2410.08260/) (2024)
- [Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer](https://mlsanity.com/paper/2412.00733/) (2024)
- [Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatars](https://mlsanity.com/paper/2008.10174/) (2020)
- [Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animation](https://mlsanity.com/paper/2410.07718/) (2024)

## Built on by (most cited first, 19 of 19)

- [Distribution Matching Distillation Meets Reinforcement Learning](https://mlsanity.com/paper/2511.13649/) (2025)
- [Phased One-Step Adversarial Equilibrium for Video Diffusion Models](https://mlsanity.com/paper/2508.21019/) (2025)
- [OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation](https://mlsanity.com/paper/2604.11804/) (2026)
- [Efficient Video Diffusion Models: Advancements and Challenges](https://mlsanity.com/paper/2604.15911/) (2026)
- [MaineCoon: Pursuing A Real-Time Audio-Visual Social World Model](https://mlsanity.com/paper/2606.17800/) (2026)
- [FastLightGen: Fast and Light Video Generation with Fewer Steps and Parameters](https://mlsanity.com/paper/2603.01685/) (2026)
- [FlashMotion: Few-Step Controllable Video Generation with Trajectory Guidance](https://mlsanity.com/paper/2603.12146/) (2026)
- [AAD-1: Asymmetric Adversarial Distillation for One-Step Autoregressive Video Generation](https://mlsanity.com/paper/2606.03972/) (2026)
- [Guidance Matters: Rethinking the Evaluation Pitfall for Text-to-Image Generation](https://mlsanity.com/paper/2602.22570/) (2026)
- [CRAFT: Aligning Diffusion Models with Fine-Tuning Is Easier Than You Think](https://mlsanity.com/paper/2603.18991/) (2026)
- [LIVEditor-14B: Lightning Unified Video Editing via In-Context Sparse Attention](https://mlsanity.com/paper/2605.04569/) (2026)
- [From Structure to Detail: Hierarchical Distillation for Efficient Diffusion Model](https://mlsanity.com/paper/2511.08930/) (2025)
- [MobileWan: Closing the Quality Gap for Mobile Video Diffusion](https://mlsanity.com/paper/2607.06173/) (2026)
- [PixelWizard: Towards Efficient High-Fidelity Video Generation at Ultra-Large Spatial Resolution](https://mlsanity.com/paper/2605.25801/) (2026)
- [Late-to-Early Training: LET LLMs Learn Earlier, So Faster and Better](https://mlsanity.com/paper/2602.05393/) (2026)
- [Joint Alignment and Distillation for Video Generation via Sample-Guided Distribution Matching](https://mlsanity.com/paper/2609.04283/) (2026)
- [WaDi: Weight Direction-aware Distillation for One-step Image Synthesis](https://mlsanity.com/paper/2603.08258/) (2026)
- [PDMD: Projected Distribution Matching Distillation for Video Diffusion Models](https://mlsanity.com/paper/2609.35768/) (2026)
- [Golden Anchor for Video Generation](https://mlsanity.com/paper/2610.13217/) (2026)

## Abstract (arXiv)

Recently, open-source video diffusion models (VDMs), such as WanX, Magic141 and HunyuanVideo, have been scaled to over 10 billion parameters. These large-scale VDMs have demonstrated significant improvements over smaller-scale VDMs across multiple dimensions, including enhanced visual quality and more natural motion dynamics. However, these models face two major limitations: (1) High inference overhead: Large-scale VDMs require approximately 10 minutes to synthesize a 28-step video on a single H100 GPU. (2) Limited in portrait video synthesis: Models like WanX-I2V and HunyuanVideo-I2V often produce unnatural facial expressions and movements in portrait videos. To address these challenges, we propose MagicDistillation, a novel framework designed to reduce inference overhead while ensuring the generalization of VDMs for portrait video synthesis. Specifically, we primarily use sufficiently high-quality talking video to fine-tune Magic141, which is dedicated to portrait video synthesis. We then employ LoRA to effectively and efficiently fine-tune the fake DiT within the step distillation framework known as distribution matching distillation (DMD). Following this, we apply weak-to-strong (W2S) distribution matching and minimize the discrepancy between the fake data distribution and the ground truth distribution, thereby improving the visual fidelity and motion dynamics of the synthesized videos. Experimental results on portrait video synthesis demonstrate the effectiveness of MagicDistillation, as our method surpasses Euler, LCM, and DMD baselines in both FID/FVD metrics and VBench. Moreover, MagicDistillation, requiring only 4 steps, also outperforms WanX-I2V (14B) and HunyuanVideo-I2V (13B) on visualization and VBench. Our project page is https://magicdistillation.github.io/MagicDistillation/.