# Deep Equilibrium Architectures for Inverse Problems in Imaging

arXiv: https://arxiv.org/abs/2102.07944 · HTML: https://mlsanity.com/paper/2102.07944/

## Key facts

- Cited by 127 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 40 references that are in the map.
- First posted on arXiv 16 Feb 2021.

## Builds on (most cited first, 26 of 26)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [MoDL: Model Based Deep Learning Architecture for Inverse Problems](https://mlsanity.com/paper/1712.02862/) (2017)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [The Little Engine that Could: Regularization by Denoising (RED)](https://mlsanity.com/paper/1611.02862/) (2016)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Learned Primal-dual Reconstruction](https://mlsanity.com/paper/1707.06474/) (2017)
- [Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applications](https://mlsanity.com/paper/1605.01710/) (2016)
- [On instabilities of deep learning in image reconstruction - Does AI come at a cost?](https://mlsanity.com/paper/1902.05300/) (2019)
- [Plug-and-Play Methods Provably Converge with Properly Trained Denoisers](https://mlsanity.com/paper/1905.05406/) (2019)
- [Implicit Deep Learning](https://mlsanity.com/paper/1908.06315/) (2019)
- [Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems](https://mlsanity.com/paper/1704.03488/) (2017)
- [RARE: Image Reconstruction using Deep Priors Learned without Ground Truth](https://mlsanity.com/paper/1912.05854/) (2019)
- [Solving Inverse Problems With Deep Neural Networks -- Robustness Included?](https://mlsanity.com/paper/2011.04268/) (2020)
- [Trellis Networks for Sequence Modeling](https://mlsanity.com/paper/1810.06682/) (2018)
- [Neural Proximal Gradient Descent for Compressive Imaging](https://mlsanity.com/paper/1806.03963/) (2018)
- [Recurrent Stacking of Layers for Compact Neural Machine Translation Models](https://mlsanity.com/paper/1807.05353/) (2018)
- [Momentum-Net: Fast and convergent iterative neural network for inverse problems](https://mlsanity.com/paper/1907.11818/) (2019)
- [Improving Robustness of Deep-Learning-Based Image Reconstruction](https://mlsanity.com/paper/2002.11821/) (2020)
- [Results of the 2020 fastMRI Challenge for Machine Learning MR Image Reconstruction](https://mlsanity.com/paper/2012.06318/) (2020)
- [Super-Resolution via Image-Adapted Denoising CNNs: Incorporating External and Internal Learning](https://mlsanity.com/paper/1811.12866/) (2018)
- [Accelerating ADMM for Efficient Simulation and Optimization](https://mlsanity.com/paper/1909.00470/) (2019)

## Built on by (most cited first, 30 of 127)

- [JFB: Jacobian-Free Backpropagation for Implicit Networks](https://mlsanity.com/paper/2103.12803/) (2021)
- [DMPlug: A Plug-in Method for Solving Inverse Problems with Diffusion Models](https://mlsanity.com/paper/2405.16749/) (2024)
- [Physics-Driven Deep Learning for Computational Magnetic Resonance Imaging](https://mlsanity.com/paper/2203.12215/) (2022)
- [Learned reconstruction methods with convergence guarantees](https://mlsanity.com/paper/2206.05431/) (2022)
- [A Neural-Network-Based Convex Regularizer for Inverse Problems](https://mlsanity.com/paper/2211.12461/) (2022)
- [HUMUS-Net: Hybrid unrolled multi-scale network architecture for accelerated MRI reconstruction](https://mlsanity.com/paper/2203.08213/) (2022)
- [SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit models](https://mlsanity.com/paper/2106.00553/) (2021)
- [Learning the optimal Tikhonov regularizer for inverse problems](https://mlsanity.com/paper/2106.06513/) (2021)
- [What's in a Prior? Learned Proximal Networks for Inverse Problems](https://mlsanity.com/paper/2310.14344/) (2023)
- [Recovery Analysis for Plug-and-Play Priors using the Restricted Eigenvalue Condition](https://mlsanity.com/paper/2106.03668/) (2021)
- [Online Deep Equilibrium Learning for Regularization by Denoising](https://mlsanity.com/paper/2205.13051/) (2022)
- [TorchDEQ: A Library for Deep Equilibrium Models](https://mlsanity.com/paper/2310.18605/) (2023)
- [Fixed Point Diffusion Models](https://mlsanity.com/paper/2401.08741/) (2024)
- [Robust Inverse Framework using Knowledge-guided Self-Supervised Learning: An application to Hydrology](https://mlsanity.com/paper/2109.06429/) (2021)
- [Joint inference and input optimization in equilibrium networks](https://mlsanity.com/paper/2111.13236/) (2021)
- [Explainable AI via Learning to Optimize](https://mlsanity.com/paper/2204.14174/) (2022)
- [Near-Exact Recovery for Tomographic Inverse Problems via Deep Learning](https://mlsanity.com/paper/2206.07050/) (2022)
- [Differentiating Through Integer Linear Programs with Quadratic Regularization and Davis-Yin Splitting](https://mlsanity.com/paper/2301.13395/) (2023)
- [Prior Mismatch and Adaptation in PnP-ADMM with a Nonconvex Convergence Analysis](https://mlsanity.com/paper/2310.00133/) (2023)
- [Global Convergence of Over-parameterized Deep Equilibrium Models](https://mlsanity.com/paper/2205.13814/) (2022)
- [Lyapunov-Stable Deep Equilibrium Models](https://mlsanity.com/paper/2304.12707/) (2023)
- [Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models](https://mlsanity.com/paper/2309.06642/) (2023)
- [PtychoDV: Vision Transformer-Based Deep Unrolling Network for Ptychographic Image Reconstruction](https://mlsanity.com/paper/2310.07504/) (2023)
- [Deep equilibrium networks are sensitive to initialization statistics](https://mlsanity.com/paper/2207.09432/) (2022)
- [Deep Model-Based Architectures for Inverse Problems under Mismatched Priors](https://mlsanity.com/paper/2207.13200/) (2022)
- [Block Coordinate Plug-and-Play Methods for Blind Inverse Problems](https://mlsanity.com/paper/2305.12672/) (2023)
- [Self-Supervised Scalable Deep Compressed Sensing](https://mlsanity.com/paper/2308.13777/) (2023)
- [Memory-efficient model-based deep learning with convergence and robustness guarantees](https://mlsanity.com/paper/2206.04797/) (2022)
- [PINQI: An End-to-End Physics-Informed Approach to Learned Quantitative MRI Reconstruction](https://mlsanity.com/paper/2306.11023/) (2023)
- [Unfolded proximal neural networks for robust image Gaussian denoising](https://mlsanity.com/paper/2308.03139/) (2023)

## Abstract (arXiv)

Recent efforts on solving inverse problems in imaging via deep neural networks use architectures inspired by a fixed number of iterations of an optimization method. The number of iterations is typically quite small due to difficulties in training networks corresponding to more iterations; the resulting solvers cannot be run for more iterations at test time without incurring significant errors. This paper describes an alternative approach corresponding to an infinite number of iterations, yielding a consistent improvement in reconstruction accuracy above state-of-the-art alternatives and where the computational budget can be selected at test time to optimize context-dependent trade-offs between accuracy and computation. The proposed approach leverages ideas from Deep Equilibrium Models, where the fixed-point iteration is constructed to incorporate a known forward model and insights from classical optimization-based reconstruction methods.