# Deep Equilibrium Approaches to Diffusion Models

arXiv: https://arxiv.org/abs/2210.12867 · HTML: https://mlsanity.com/paper/2210.12867/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 59 of its 79 references that are in the map.
- First posted on arXiv 23 Oct 2022.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 59)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Diffusion Models for Adversarial Purification](https://mlsanity.com/paper/2205.07460/) (2022)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.14951/) (2021)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)
- [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016)
- [Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed](https://mlsanity.com/paper/2101.02388/) (2021)
- [Compressed Sensing using Generative Models](https://mlsanity.com/paper/1703.03208/) (2017)
- [Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction](https://mlsanity.com/paper/2112.05146/) (2021)

## Built on by (most cited first, 30 of 42)

- [Consistency Models Made Easy](https://mlsanity.com/paper/2406.14548/) (2024)
- [One-Step Diffusion Distillation through Score Implicit Matching](https://mlsanity.com/paper/2410.16794/) (2024)
- [Towards Coherent Image Inpainting Using Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2304.03322/) (2023)
- [One-Step Diffusion Distillation via Deep Equilibrium Models](https://mlsanity.com/paper/2401.08639/) (2023)
- [Efficient Diffusion Models: A Survey](https://mlsanity.com/paper/2502.06805/) (2025)
- [Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferences](https://mlsanity.com/paper/2410.18881/) (2024)
- [ExposureDiffusion: Learning to Expose for Low-light Image Enhancement](https://mlsanity.com/paper/2307.07710/) (2023)
- [Flow Generator Matching](https://mlsanity.com/paper/2410.19310/) (2024)
- [TorchDEQ: A Library for Deep Equilibrium Models](https://mlsanity.com/paper/2310.18605/) (2023)
- [Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach](https://mlsanity.com/paper/2401.15652/) (2024)
- [Deep Equilibrium Diffusion Restoration with Parallel Sampling](https://mlsanity.com/paper/2311.11600/) (2023)
- [Fixed Point Diffusion Models](https://mlsanity.com/paper/2401.08741/) (2024)
- [ELT: Elastic Looped Transformers for Visual Generation](https://mlsanity.com/paper/2604.09168/) (2026)
- [AdjointDPM: Adjoint Sensitivity Method for Gradient Backpropagation of Diffusion Probabilistic Models](https://mlsanity.com/paper/2307.10711/) (2023)
- [Diffusion Models are Secretly Exchangeable: Parallelizing DDPMs via Autospeculation](https://mlsanity.com/paper/2505.03983/) (2025)
- [David and Goliath: Small One-step Model Beats Large Diffusion with Score Post-training](https://mlsanity.com/paper/2410.20898/) (2024)
- [Positive concave deep equilibrium models](https://mlsanity.com/paper/2402.04029/) (2024)
- [Nested Diffusion Processes for Anytime Image Generation](https://mlsanity.com/paper/2305.19066/) (2023)
- [Deep Implicit Optimization enables Robust Learnable Features for Deformable Image Registration](https://mlsanity.com/paper/2406.07361/) (2024)
- [One-Way Ticket:Time-Independent Unified Encoder for Distilling Text-to-Image Diffusion Models](https://mlsanity.com/paper/2505.21960/) (2025)
- [Reversible Deep Equilibrium Models](https://mlsanity.com/paper/2509.12917/) (2025)
- [Deep equilibrium models as estimators for continuous latent variables](https://mlsanity.com/paper/2211.05943/) (2022)
- [Deep Equilibrium Multimodal Fusion](https://mlsanity.com/paper/2306.16645/) (2023)
- [Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling](https://mlsanity.com/paper/2410.01440/) (2024)
- [Subhomogeneous Deep Equilibrium Models](https://mlsanity.com/paper/2403.00720/) (2024)
- [Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspective](https://mlsanity.com/paper/2410.23391/) (2024)
- [Improving Adversarial Robustness of DEQs with Explicit Regulations Along the Neural Dynamics](https://mlsanity.com/paper/2306.01435/) (2023)
- [From Darkness to Detail: Frequency-Aware SSMs for Low-Light Vision](https://mlsanity.com/paper/2408.09650/) (2024)
- [Quantum Deep Equilibrium Models](https://mlsanity.com/paper/2410.23940/) (2024)
- [Certified Robustness for Deep Equilibrium Models via Serialized Random Smoothing](https://mlsanity.com/paper/2411.00899/) (2024)

## Abstract (arXiv)

Diffusion-based generative models are extremely effective in generating high-quality images, with generated samples often surpassing the quality of those produced by other models under several metrics. One distinguishing feature of these models, however, is that they typically require long sampling chains to produce high-fidelity images. This presents a challenge not only from the lenses of sampling time, but also from the inherent difficulty in backpropagating through these chains in order to accomplish tasks such as model inversion, i.e. approximately finding latent states that generate known images. In this paper, we look at diffusion models through a different perspective, that of a (deep) equilibrium (DEQ) fixed point model. Specifically, we extend the recent denoising diffusion implicit model (DDIM; Song et al. 2020), and model the entire sampling chain as a joint, multivariate fixed point system. This setup provides an elegant unification of diffusion and equilibrium models, and shows benefits in 1) single image sampling, as it replaces the fully-serial typical sampling process with a parallel one; and 2) model inversion, where we can leverage fast gradients in the DEQ setting to much more quickly find the noise that generates a given image. The approach is also orthogonal and thus complementary to other methods used to reduce the sampling time, or improve model inversion. We demonstrate our method's strong performance across several datasets, including CIFAR10, CelebA, and LSUN Bedrooms and Churches.