mlsanityData to 14 Oct 2026

One-Step Diffusion Distillation via Deep Equilibrium Models

arXiv 2401.08639 ↗First posted 12 Dec 2023cs.CV cs.LGcited by 61

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models excel at producing high-quality samples but naively require hundreds of iterations, prompting multiple attempts to distill the generation process into a faster network. However, many existing approaches suffer from a variety of challenges: the process for distillation training can be complex, often requiring multiple training stages, and the resulting models perform poorly when utilized in single-step generative applications. In this paper, we introduce a simple yet effective means of distilling diffusion models directly from initial noise to the resulting image. Of particular importance to our approach is to leverage a new Deep Equilibrium (DEQ) model as the distilled architecture: the Generative Equilibrium Transformer (GET). Our method enables fully offline training with just noise/image pairs from the diffusion model while achieving superior performance compared to existing one-step methods on comparable training budgets. We demonstrate that the DEQ architecture is crucial to this capability, as GET matches a 5× larger ViT in terms of FID scores while striking a critical balance of computational cost and image quality. Code, checkpoints, and datasets are available.

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 100 of its 115 references that are in the map. First posted on arXiv 12 Dec 2023.

Builds on

100 of 115 references.

Closest ancestors · 29

Same direction, newest first

  1. 2023TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation7 Mar
  2. 2023Consistency Models2 Mar
  3. 2022Scalable Diffusion Models with Transformers19 Dec
  4. 2022Refining Generative Process with Discriminator Guidance in Score-based Diffusion Models28 Nov
  5. 2022Fast Sampling of Diffusion Models via Operator Learning24 Nov
  6. 2022Deep Equilibrium Approaches to Diffusion Models23 Oct
  7. 2022GENIE: Higher-Order Denoising Diffusion Solvers11 Oct
  8. 2022On Distillation of Guided Diffusion Models6 Oct
  9. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models15 Jun
  10. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  11. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  12. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
17 more
  1. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  2. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models17 Jan
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  4. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  5. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusion14 Dec
  6. 2021Variational Diffusion Models1 Jul
  7. 2021Score-based Generative Modeling in Latent Space10 Jun
  8. 2021Learning to Efficiently Sample from Diffusion Probabilistic Models7 Jun
  9. 2021On Fast Sampling of Diffusion Probabilistic Models31 May
  10. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
  11. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  12. 2021Noise Estimation for Generative Diffusion Models6 Apr
  13. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  14. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  15. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  16. 2020Denoising Diffusion Implicit Models6 Oct
  17. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 71

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2023GPT-4 Technical Reportcited by 18,396
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  9. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  10. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  12. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
38 more
  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2016Layer Normalizationcited by 5,669
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  9. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  11. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  12. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  13. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  14. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  15. 2016Improved Techniques for Training GANscited by 2,970
  16. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  17. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  18. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  19. 2022Video Diffusion Modelscited by 2,305
  20. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  21. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  22. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  23. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  24. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  25. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  26. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  27. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  28. 2020On Layer Normalization in the Transformer Architecturecited by 922
  29. 2016Training Deep Nets with Sublinear Memory Costcited by 848
  30. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 677
  31. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 647
  32. 2019Deep Equilibrium Modelscited by 615
  33. 2022GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generationcited by 516
  34. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 492
  35. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 450
  36. 2022DiffDock: Diffusion Steps, Twists, and Turns for Molecular Dockingcited by 313
  37. 2019Stabilizing Transformers for Reinforcement Learningcited by 270
  38. 2023Noise2Music: Text-conditioned Music Generation with Diffusion Modelscited by 173

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 20; 2025: 20; 2026: 21.

61 papers.

Sort:

By area:

  1. 2025Mean Flows for One-step Generative Modelingcited by 561
  2. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generationcited by 214
  3. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  4. 2024Consistency Models Made Easycited by 144
  5. 2024DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splattingcited by 82
  6. 2024One-Step Diffusion Distillation through Score Implicit Matchingcited by 71
  7. 2024Inference-Time Alignment of Diffusion Models with Direct Noise Optimizationcited by 55
  8. 2024Adversarial Score identity Distillation: Rapidly Surpassing the Teacher in One Stepcited by 47
  9. 2024Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Seriescited by 43
  10. 2024Reward Guided Latent Consistency Distillationcited by 35
  11. 2024Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferencescited by 31
  12. 2024Accelerating Parallel Sampling of Diffusion Modelscited by 30
38 more
  1. 2024Flow Generator Matchingcited by 26
  2. 2024Unleashing the Power of One-Step Diffusion based Image Super-Resolution via a Large-Scale Diffusion Discriminatorcited by 19
  3. 2024Fixed Point Diffusion Modelscited by 16
  4. 2025Ultra-Fast Language Generation via Discrete Diffusion Divergence Instructcited by 16
  5. 2025REWIND: Real-Time Egocentric Whole-Body Motion Diffusion with Exemplar-Based Identity Conditioningcited by 14
  6. 2026ELT: Elastic Looped Transformers for Visual Generationcited by 14
  7. 2024Physics Informed Distillation for Diffusion Modelscited by 13
  8. 2025AB-Cache: Training-Free Acceleration of Diffusion Models via Adams-Bashforth Cached Feature Reusecited by 13
  9. 2025Uni-Instruct: One-step Diffusion Model through Unified Diffusion Divergence Instructioncited by 13
  10. 2025Learning an Image Editing Model without Image Editing Pairscited by 13
  11. 2025SoFlow: Solution Flow Models for One-Step Generative Modelingcited by 13
  12. 2024David and Goliath: Small One-step Model Beats Large Diffusion with Score Post-trainingcited by 9
  13. 2025One-Step Offline Distillation of Diffusion-based Models via Koopman Modelingcited by 9
  14. 2024Hierarchical VAE with a Diffusion-based VampPriorcited by 8
  15. 2025Modular MeanFlow: Towards Stable and Scalable One-Step Generative Modelingcited by 8
  16. 2025Learning Straight Flows: Variational Flow Matching for Efficient Generationcited by 8
  17. 2024Positive concave deep equilibrium modelscited by 7
  18. 2026LeapAlign: Post-Training Flow Matching Models at Any Generation Step by Building Two-Step Trajectoriescited by 6
  19. 2025One-Way Ticket:Time-Independent Unified Encoder for Distilling Text-to-Image Diffusion Modelscited by 5
  20. 2025Universal Inverse Distillation for Matching Models with Real-Data Supervision (No GANs)cited by 5
  21. 2025Functional Mean Flow in Hilbert Spacecited by 5
  22. 2026Fixed-Point Masked Generative Modelingcited by 5
  23. 2024Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modelingcited by 4
  24. 2025On the Design of One-step Diffusion via Shortcutting Flow Pathscited by 4
  25. 2026ReFORM: Reflected Flows for On-support Offline RL via Noise Manipulationcited by 4
  26. 2026AR Forcing: Towards Long-Horizon Robot Navigation World Modelcited by 4
  27. 2025DEQuify your force field: More efficient simulations using deep equilibrium modelscited by 3
  28. 2025Hierarchical Schedule Optimization for Fast and Robust Diffusion Model Samplingcited by 3
  29. 2026RMFlow: Refined Mean Flow by a Noise-Injection Step for Multimodal Generationcited by 3
  30. 2026COT-FM: Cluster-wise Optimal Transport Flow Matchingcited by 3
  31. 2025Distributional Statistics Restore Training Data Auditability in One-step Distilled Diffusion Modelscited by 2
  32. 2026Trajectory Consistency for One-Step Generation on Euler Mean Flowscited by 2
  33. 2026Transition Flow Matchingcited by 2
  34. 2026Drift Flow Matchingcited by 2
  35. 2026Recursive Scaling in Masked Diffusion Modelscited by 2
  36. 2025Adaptive Discretization for Consistency Modelscited by 1
  37. 2026Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approachcited by 1
  38. 2026Accelerated Sequential Flow Matching: A Bayesian Filtering Perspectivecited by 1

50 most cited of 61 All 61 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Diffusion Models Are Innate One-Step Generators
  2. 2024Simple and Fast Distillation of Diffusion Models
  3. 2025Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation
  4. 2025A Survey on Pre-Trained Diffusion Model Distillations
  5. 2026EMPURPLE: A Free Lunch for Diffusion Distillation based on the Information Bottleneck
  6. 2024SFDDM: Single-fold Distillation for Diffusion models
  7. 2023One-step Diffusion with Distribution Matching Distillation
  8. 2026Consistent Distribution Matching for Data-Free Diffusion Distillation