mlsanityData to 7 Oct 2026

On Distillation of Guided Diffusion Models

arXiv 2210.03142 ↗First posted 6 Oct 2022cs.CV cs.AI cs.LGcited by 721

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Classifier-free guided diffusion models have recently been shown to be highly effective at high-resolution image generation, and they have been widely used in large-scale diffusion frameworks including DALLE-2, Stable Diffusion and Imagen. However, a downside of classifier-free guided diffusion models is that they are computationally expensive at inference time since they require evaluating two diffusion models, a class-conditional model and an unconditional model, tens to hundreds of times. To deal with this limitation, we propose an approach to distilling classifier-free guided diffusion models into models that are fast to sample from: Given a pre-trained classifier-free guided model, we first learn a single model to match the output of the combined conditional and unconditional models, and then we progressively distill that model to a diffusion model that requires much fewer sampling steps. For standard diffusion models trained on the pixel-space, our approach is able to generate images visually comparable to that of the original model using as few as 4 sampling steps on ImageNet 64x64 and CIFAR-10, achieving FID/IS scores comparable to that of the original model while being up to 256 times faster to sample from. For diffusion models trained on the latent-space (e.g., Stable Diffusion), our approach is able to generate high-fidelity images using as few as 1 to 4 denoising steps, accelerating inference by at least 10-fold compared to existing methods on ImageNet 256x256 and LAION datasets. We further demonstrate the effectiveness of our approach on text-guided image editing and inpainting, where our distilled model is able to generate high-quality results using as few as 2-4 denoising steps.

Key facts. Cited by 721 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 35 of its 49 references that are in the map. First posted on arXiv 6 Oct 2022.

Builds on

35 of 49 references.

Closest ancestors · 16

Same direction, newest first

  1. 2022Classifier-Free Diffusion Guidance26 Jul
  2. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  3. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  4. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  5. 2022Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality11 Feb
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  7. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representation30 Nov
  8. 2021Variational Diffusion Models1 Jul
  9. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  10. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
  11. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  12. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
4 more
  1. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  3. 2020Denoising Diffusion Implicit Models6 Oct
  4. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 19

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  9. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  11. 2016Improved Techniques for Training GANscited by 2,938
  12. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
7 more
  1. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  2. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  3. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  4. 2022GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generationcited by 511
  5. 2022Dual Diffusion Implicit Bridges for Image-to-Image Translationcited by 212
  6. 2021Symbolic Music Generation with Diffusion Modelscited by 152
  7. 2021D2C: Diffusion-Denoising Models for Few-shot Conditional Generationcited by 96

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 136; 2024: 230; 2025: 219; 2026: 126.

721 papers.

Sort:

By area:

  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  2. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  3. 2023Consistency Modelscited by 1,778
  4. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  5. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  6. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  7. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  8. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  9. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  10. 2023Adversarial Diffusion Distillationcited by 741
  11. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  12. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
38 more
  1. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusioncited by 448
  2. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
  3. 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 413
  4. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  5. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  6. 2024One Step Diffusion via Shortcut Modelscited by 373
  7. 2023DeepCache: Accelerating Diffusion Models for Freecited by 346
  8. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  9. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
  10. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 276
  11. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
  12. 2023Q-Diffusion: Quantizing Diffusion Modelscited by 266
  13. 2023Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylizationcited by 251
  14. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Modelcited by 245
  15. 2023Efficient Diffusion Training via Min-SNR Weighting Strategycited by 242
  16. 2023SinSR: Diffusion-Based Image Super-Resolution in a Single Stepcited by 235
  17. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  18. 2023StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesiscited by 212
  19. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  20. 2023LCM-LoRA: A Universal Stable-Diffusion Acceleration Modulecited by 211
  21. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generationcited by 211
  22. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 209
  23. 2023A Survey on Video Diffusion Modelscited by 203
  24. 2023Glaze: Protecting Artists from Style Mimicry by Text-to-Image Modelscited by 200
  25. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
  26. 2022Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Modelscited by 188
  27. 2022Fast Sampling of Diffusion Models via Operator Learningcited by 173
  28. 2023Fast Training of Diffusion Models with Masked Transformerscited by 172
  29. 2023UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANscited by 165
  30. 2023DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statisticscited by 161
  31. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 161
  32. 2024Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillationcited by 159
  33. 2024ControlNeXt: Powerful and Efficient Control for Image and Video Generationcited by 158
  34. 2023Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuningcited by 150
  35. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  36. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Cachingcited by 149
  37. 2023LEDITS++: Limitless Image Editing using Text-to-Image Modelscited by 147
  38. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learningcited by 145

50 most cited of 721 All 721 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Elucidating The Design Space of Classifier-Guided Diffusion Generation
  2. 2026Enhanced Low-Density Region Exploration in Classifier-Guided Diffusion Models Through Modified Reverse Diffusion Sampling
  3. 2023DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning
  4. 2026Revisiting Classifier-Free Guidance Methods in Latent Diffusion Models
  5. 2023Bridging the Gap: Addressing Discrepancies in Diffusion Model Training for Classifier-Free Guidance
  6. 2022Progressive Distillation for Fast Sampling of Diffusion Models
  7. 2024Multistep Distillation of Diffusion Models via Moment Matching
  8. 2024Unlocking Dataset Distillation with Diffusion Models