mlsanityData to 13 Oct 2026

One-step Diffusion with Distribution Matching Distillation

arXiv 2311.18828 ↗First posted 30 Nov 2023cs.CVcited by 991

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models generate high-quality images but require dozens of forward passes. We introduce Distribution Matching Distillation (DMD), a procedure to transform a diffusion model into a one-step image generator with minimal impact on image quality. We enforce the one-step image generator match the diffusion model at distribution level, by minimizing an approximate KL divergence whose gradient can be expressed as the difference between 2 score functions, one of the target distribution and the other of the synthetic distribution being produced by our one-step generator. The score functions are parameterized as two diffusion models trained separately on each distribution. Combined with a simple regression loss matching the large-scale structure of the multi-step diffusion outputs, our method outperforms all published few-step diffusion approaches, reaching 2.62 FID on ImageNet 64x64 and 11.49 FID on zero-shot COCO-30k, comparable to Stable Diffusion but orders of magnitude faster. Utilizing FP16 inference, our model generates images at 20 FPS on modern hardware.

Key facts. Cited by 991 papers in the map (data through 13 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 77 of its 89 references that are in the map. First posted on arXiv 30 Nov 2023.

Builds on

77 of 89 references.

Closest ancestors · 27

Same direction, newest first

  1. 2023Adversarial Diffusion Distillation28 Nov
  2. 2023LCM-LoRA: A Universal Stable-Diffusion Acceleration Module9 Nov
  3. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference6 Oct
  4. 2023Score Mismatching for Generative Modeling20 Sep
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  6. 2023BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrapping8 Jun
  7. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models29 May
  8. 2023Unifying GANs and Score-Based Diffusion as Generative Particle Models25 May
  9. 2023The Score-Difference Flow for Implicit Generative Modeling25 Apr
  10. 2023TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation7 Mar
  11. 2023Consistency Models2 Mar
  12. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models9 Feb
15 more
  1. 2022Fast Sampling of Diffusion Models via Operator Learning24 Nov
  2. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  3. 2022On Distillation of Guided Diffusion Models6 Oct
  4. 2022Classifier-Free Diffusion Guidance26 Jul
  5. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  7. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  8. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  9. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  10. 2021Variational Diffusion Models1 Jul
  11. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  12. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  13. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  14. 2020Denoising Diffusion Implicit Models6 Oct
  15. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 50

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,550
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,082
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,180
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,023
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,564
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,339
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,000
  9. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,516
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,284
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,141
  12. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,077
38 more
  1. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,607
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,605
  3. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,585
  4. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,019
  5. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,975
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,619
  7. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,768
  8. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,732
  9. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,348
  10. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,296
  11. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,289
  12. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,212
  13. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  14. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,178
  15. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 926
  16. 2016Training Deep Nets with Sublinear Memory Costcited by 840
  17. 2016Generative Adversarial Text to Image Synthesiscited by 735
  18. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 641
  19. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 621
  20. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  21. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  22. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 490
  23. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  24. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 391
  25. 2015Generative Moment Matching Networkscited by 252
  26. 2019DRIT++: Diverse Image-to-Image Translation via Disentangled Representationscited by 244
  27. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 220
  28. 2020Swapping Autoencoder for Deep Image Manipulationcited by 215
  29. 2023StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesiscited by 212
  30. 2022Diffusion-GAN: Training GANs with Diffusioncited by 204
  31. 2023UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANscited by 165
  32. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
  33. 2015Training generative neural networks via Maximum Mean Discrepancy optimizationcited by 149
  34. 2021LAFITE: Towards Language-Free Training for Text-to-Image Generationcited by 127
  35. 2021ViTGAN: Training GANs with Vision Transformerscited by 110
  36. 2019AutoGAN: Neural Architecture Search for Generative Adversarial Networkscited by 94
  37. 2020Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Searchcited by 25
  38. 2023MonoFlow: Rethinking Divergence GANs via the Perspective of Wasserstein Gradient Flowscited by 13

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 1; 2024: 141; 2025: 342; 2026: 507.

991 papers.

Sort:

By area:

  1. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 664
  2. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 590
  3. 2025Mean Flows for One-step Generative Modelingcited by 541
  4. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 414
  5. 2024One Step Diffusion via Shortcut Modelscited by 382
  6. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 327
  7. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 296
  8. 2024Diffusion Models Are Real-Time Game Enginescited by 286
  9. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 278
  10. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 277
  11. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 261
  12. 2025LongLive: Real-time Interactive Long Video Generationcited by 246
38 more
  1. 2024One-Step Effective Diffusion Network for Real-World Image Super-Resolutioncited by 245
  2. 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 221
  3. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 215
  4. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsitycited by 213
  5. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 213
  6. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generationcited by 212
  7. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 211
  8. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world modelcited by 180
  9. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 179
  10. 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 174
  11. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 165
  12. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 164
  13. 2024Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillationcited by 162
  14. 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 160
  15. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 158
  16. 2024Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesiscited by 154
  17. 2026Advancing Open-source World Modelscited by 148
  18. 2025Diffusion Adversarial Post-Training for One-Step Video Generationcited by 147
  19. 2024Consistency Models Made Easycited by 143
  20. 2025The Diffusion Dualitycited by 130
  21. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 129
  22. 2025Fast Video Generation with Sliding Tile Attentioncited by 126
  23. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 125
  24. 2024ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimizationcited by 118
  25. 2024Dirichlet Flow Matching with Applications to DNA Sequence Designcited by 113
  26. 2024One-Step Image Translation with Text-to-Image Modelscited by 113
  27. 2025Inductive Moment Matchingcited by 110
  28. 2024AlphaFold Meets Flow Matching for Generating Protein Ensemblescited by 108
  29. 2024Improving the Training of Rectified Flowscited by 107
  30. 2026Generative Modeling via Driftingcited by 106
  31. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformerscited by 104
  32. 2024PeRFlow: Piecewise Rectified Flow as Universal Plug-and-Play Acceleratorcited by 97
  33. 2024Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approachcited by 96
  34. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 95
  35. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 94
  36. 2024Align Your Steps: Optimizing Sampling Schedules in Diffusion Modelscited by 93
  37. 2025SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillationcited by 86
  38. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 86

50 most cited of 991 All 991 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Consistent Distribution Matching for Data-Free Diffusion Distillation
  2. 20261.x-Distill: Breaking the Diversity, Quality, and Efficiency Barrier in Distribution Matching Distillation
  3. 2024SFDDM: Single-fold Distillation for Diffusion models
  4. 2023One-Step Diffusion Distillation via Deep Equilibrium Models
  5. 2024Beta Sampling is All You Need: Efficient Image Generation Strategy for Diffusion Models using Stepwise Spectral Analysis
  6. 2024Non-Normal Diffusion Models
  7. 2024A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models
  8. 2024Plug-and-Play Diffusion Distillation