# Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesis

arXiv: https://arxiv.org/abs/2507.18569 · HTML: https://mlsanity.com/paper/2507.18569/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 76 of its 94 references that are in the map.
- First posted on arXiv 24 Jul 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 76)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2305.01569/) (2023)

## Built on by (most cited first, 30 of 41)

- [Seedream 4.0: Toward Next-generation Multimodal Image Generation](https://mlsanity.com/paper/2509.20427/) (2025)
- [WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling](https://mlsanity.com/paper/2512.14614/) (2025)
- [Distribution Matching Distillation Meets Reinforcement Learning](https://mlsanity.com/paper/2511.13649/) (2025)
- [Flow caching for autoregressive video generation](https://mlsanity.com/paper/2602.10825/) (2026)
- [D-OPSD: On-Policy Self-Distillation for Continuously Tuning Step-Distilled Diffusion Models](https://mlsanity.com/paper/2605.05204/) (2026)
- [AnyTalker: Scaling Multi-Person Talking Video Generation with Interactivity Refinement](https://mlsanity.com/paper/2511.23475/) (2025)
- [Diversity-Preserved Distribution Matching Distillation for Fast Visual Synthesis](https://mlsanity.com/paper/2602.03139/) (2026)
- [Flash-WAM: Modality-Aware Distillation for World Action Models](https://mlsanity.com/paper/2606.05254/) (2026)
- [Flash-DMD: Towards High-Fidelity Few-Step Image Generation with Efficient Distillation and Joint Reinforcement Learning](https://mlsanity.com/paper/2511.20549/) (2025)
- [Towards One-step Causal Video Generation via Adversarial Self-Distillation](https://mlsanity.com/paper/2511.01419/) (2025)
- [HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoising](https://mlsanity.com/paper/2603.08703/) (2026)
- [Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players](https://mlsanity.com/paper/2605.28816/) (2026)
- [Hyper-Bagel: A Unified Acceleration Framework for Multimodal Understanding and Generation](https://mlsanity.com/paper/2509.18824/) (2025)
- [Adversarial Flow Models](https://mlsanity.com/paper/2511.22475/) (2025)
- [Knot Forcing: Taming Autoregressive Video Diffusion Models for Real-time Infinite Interactive Portrait Animation](https://mlsanity.com/paper/2512.21734/) (2025)
- [A Unified View of Score-Based and Drifting Models](https://mlsanity.com/paper/2603.07514/) (2026)
- [Data-Forcing Distillation: Restoring Diversity and Fidelity in Few-Step Video Generation](https://mlsanity.com/paper/2606.18478/) (2026)
- [One-Step Flow Policy: Self-Distillation for Fast Visuomotor Policies](https://mlsanity.com/paper/2603.12480/) (2026)
- [Adaptive Video Distillation: Mitigating Oversaturation and Temporal Collapse in Few-Step Generation](https://mlsanity.com/paper/2603.21864/) (2026)
- [Efficient Video Diffusion Models: Advancements and Challenges](https://mlsanity.com/paper/2604.15911/) (2026)
- [RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO](https://mlsanity.com/paper/2605.15190/) (2026)
- [Beyond VLM-Based Rewards: Diffusion-Native Latent Reward Modeling](https://mlsanity.com/paper/2602.11146/) (2026)
- [VDOT: Efficient Unified Video Creation via Optimal Transport Distillation](https://mlsanity.com/paper/2512.06802/) (2025)
- [AAD-1: Asymmetric Adversarial Distillation for One-Step Autoregressive Video Generation](https://mlsanity.com/paper/2606.03972/) (2026)
- [1.x-Distill: Breaking the Diversity, Quality, and Efficiency Barrier in Distribution Matching Distillation](https://mlsanity.com/paper/2604.04018/) (2026)
- [Continuous-Time Distribution Matching for Few-Step Diffusion Distillation](https://mlsanity.com/paper/2605.06376/) (2026)
- [Distribution Matching Distillation without Fake Score Network](https://mlsanity.com/paper/2605.19256/) (2026)
- [Curvature-Adaptive Consistency Flow Matching: Autonomous Trajectory Optimization via Reinforcement Learning](https://mlsanity.com/paper/2606.22394/) (2026)
- [DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation](https://mlsanity.com/paper/2607.26811/) (2026)
- [AdvFD: Boosting Visual Generation via Adversarial Fr'echet Distance Loss](https://mlsanity.com/paper/2608.11205/) (2026)

## Abstract (arXiv)

Distribution Matching Distillation (DMD) is a promising score distillation technique that compresses pre-trained teacher diffusion models into efficient one-step or multi-step student generators. Nevertheless, its reliance on the reverse Kullback-Leibler (KL) divergence minimization potentially induces mode collapse (or mode-seeking) in certain applications. To circumvent this inherent drawback, we propose Adversarial Distribution Matching (ADM), a novel framework that leverages diffusion-based discriminators to align the latent predictions between real and fake score estimators for score distillation in an adversarial manner. In the context of extremely challenging one-step distillation, we further improve the pre-trained generator by adversarial distillation with hybrid discriminators in both latent and pixel spaces. Different from the mean squared error used in DMD2 pre-training, our method incorporates the distributional loss on ODE pairs collected from the teacher model, and thus providing a better initialization for score distillation fine-tuning in the next stage. By combining the adversarial distillation pre-training with ADM fine-tuning into a unified pipeline termed DMDX, our proposed method achieves superior one-step performance on SDXL compared to DMD2 while consuming less GPU time. Additional experiments that apply multi-step ADM distillation on SD3-Medium, SD3.5-Large, and CogVideoX set a new benchmark towards efficient image and video synthesis.