mlsanityData to 14 Oct 2026

Online Deep Equilibrium Learning for Regularization by Denoising

arXiv 2205.13051 ↗First posted 25 May 2022eess.IV cs.CV cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

Plug-and-Play Priors (PnP) and Regularization by Denoising (RED) are widely-used frameworks for solving imaging inverse problems by computing fixed-points of operators combining physical measurement models and learned image priors. While traditional PnP/RED formulations have focused on priors specified using image denoisers, there is a growing interest in learning PnP/RED priors that are end-to-end optimal. The recent Deep Equilibrium Models (DEQ) framework has enabled memory-efficient end-to-end learning of PnP/RED priors by implicitly differentiating through the fixed-point equations without storing intermediate activation values. However, the dependence of the computational/memory complexity of the measurement models in PnP/RED on the total number of measurements leaves DEQ impractical for many imaging applications. We propose ODER as a new strategy for improving the efficiency of DEQ through stochastic approximations of the measurement models. We theoretically analyze ODER giving insights into its convergence and ability to approximate the traditional DEQ approach. Our numerical results suggest the potential improvements in training/testing complexity due to ODER on three distinct imaging applications.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 64 of its 115 references that are in the map. First posted on arXiv 25 May 2022.

Builds on

64 of 115 references.

References · 64

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2018Neural Ordinary Differential Equationscited by 3,779
  4. 2021Convex optimizationcited by 2,515
  5. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  6. 2018Group Normalizationcited by 1,524
  7. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,358
  8. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  9. 2015Cyclical Learning Rates for Training Neural Networkscited by 629
  10. 2019Deep Equilibrium Modelscited by 615
  11. 2020Plug-and-Play Image Restoration with Deep Denoiser Priorcited by 470
  12. 2016Deep Convolutional Neural Network for Inverse Problems in Imagingcited by 433
38 more
  1. 2019Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processingcited by 421
  2. 2017Learning Deep CNN Denoiser Prior for Image Restorationcited by 408
  3. 2019Augmented Neural ODEscited by 397
  4. 2017MoDL: Model Based Deep Learning Architecture for Inverse Problemscited by 387
  5. 2016The Little Engine that Could: Regularization by Denoising (RED)cited by 307
  6. 2020Deep Learning Techniques for Inverse Problems in Imagingcited by 291
  7. 2017Learned Primal-dual Reconstructioncited by 251
  8. 2016Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applicationscited by 246
  9. 2017Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)cited by 224
  10. 2019Plug-and-Play Methods Provably Converge with Properly Trained Denoiserscited by 198
  11. 2017Non-convex Optimization for Machine Learningcited by 133
  12. 2021Deep Equilibrium Architectures for Inverse Problems in Imagingcited by 127
  13. 2019Deep Plug-and-Play Super-Resolution for Arbitrary Blur Kernelscited by 112
  14. 2021Gradient Step Denoiser for convergent Plug-and-Playcited by 108
  15. 2017Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problemscited by 107
  16. 2018Regularization by Denoising: Clarifications and New Interpretationscited by 103
  17. 2020Monotone operator equilibrium networkscited by 100
  18. 2017Framing U-Net via Deep Convolutional Framelets: Application to Sparse-view CTcited by 88
  19. 2016A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstructioncited by 87
  20. 2021JFB: Jacobian-Free Backpropagation for Implicit Networkscited by 86
  21. 2019DeepRED: Deep Image Prior Powered by REDcited by 85
  22. 2021Equivariant Imaging: Learning Beyond the Range Spacecited by 83
  23. 2017A Review of Convolutional Neural Networks for Inverse Problems in Imagingcited by 81
  24. 2020Learning Differential Equations that are Easy to Solvecited by 81
  25. 2018Denoising Prior Driven Deep Neural Network for Image Restorationcited by 80
  26. 2015Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolationcited by 78
  27. 2017CNN-Based Projected Gradient Descent for Consistent Image Reconstructioncited by 77
  28. 2019Plug and play methods for magnetic resonance imaging (long version)cited by 75
  29. 2018An Online Plug-and-Play Algorithm for Regularized Image Reconstructioncited by 70
  30. 2019RARE: Image Reconstruction using Deep Priors Learned without Ground Truthcited by 64
  31. 2017Plug-and-Play Unplugged: Optimization Free Reconstruction using Consensus Equilibriumcited by 58
  32. 2020Regularization by Denoising via Fixed-Point Projection (RED-PRO)cited by 55
  33. 2020Scalable Plug-and-Play ADMM with Convergence Guaranteescited by 50
  34. 2018prDeep: Robust Phase Retrieval with a Flexible Deep Networkcited by 47
  35. 2017Model based learning for accelerated, limited-view 3D photoacoustic tomographycited by 45
  36. 2020Provable Convergence of Plug-and-Play Priors with MMSE denoiserscited by 39
  37. 2020Tuning-free Plug-and-Play Proximal Algorithm for Inverse Imaging Problemscited by 38
  38. 2021On the Theory of Implicit Deep Learning: Global Convergence with Implicit Layerscited by 35

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 8; 2024: 5; 2025: 3; 2026: 3.

22 papers.

Sort:

By area:

  1. 2022DOLCE: A Model-Based Probabilistic Diffusion Framework for Limited-Angle CT Reconstructioncited by 74
  2. 2023What's in a Prior? Learned Proximal Networks for Inverse Problemscited by 25
  3. 2023TorchDEQ: A Library for Deep Equilibrium Modelscited by 21
  4. 2024Learning Image Priors through Patch-based Diffusion Models for Solving Inverse Problemscited by 20
  5. 2024DiffusionBlend: Learning 3D Image Prior through Position-aware Diffusion Score Blending for 3D Computed Tomography Reconstructioncited by 13
  6. 2023Differentiating Through Integer Linear Programs with Quadratic Regularization and Davis-Yin Splittingcited by 11
  7. 2023Prior Mismatch and Adaptation in PnP-ADMM with a Nonconvex Convergence Analysiscited by 11
  8. 2023PtychoDV: Vision Transformer-Based Deep Unrolling Network for Ptychographic Image Reconstructioncited by 9
  9. 2023Block Coordinate Plug-and-Play Methods for Blind Inverse Problemscited by 8
  10. 2023Self-Supervised Scalable Deep Compressed Sensingcited by 8
  11. 2025Measurement Score-Based Diffusion Modelcited by 8
  12. 2024On Logical Extrapolation for Mazes with Recurrent and Implicit Networkscited by 7
10 more
  1. 2022Robustness of Deep Equilibrium Architectures to Changes in the Measurement Modelcited by 4
  2. 2024Overcoming Distribution Shifts in Plug-and-Play Methods with Test-Time Trainingcited by 3
  3. 2024Swap-Net: A Memory-Efficient 2.5D Network for Sparse-View 3D Cone Beam CT Reconstructioncited by 3
  4. 2025End-to-End Training of High-Dimensional Optimal Control with Implicit Hamiltonians via Jacobian-Free Backpropagationcited by 3
  5. 2023Efficient Model-Based Deep Learning via Network Pruning and Fine-Tuningcited by 2
  6. 2025Recovering Pulse Waves from Video Using Deep Unrolling and Deep Equilibrium Modelscited by 2
  7. 2022Self-Supervised Deep Equilibrium Models for Inverse Problems with Theoretical Guaranteescited by 1
  8. 2026Efficient Unrolled Networks for Large-Scale 3D Inverse Problemscited by 1
  9. 2026On the Convergence of Jacobian-Free Backpropagation for Optimal Control Problems with Implicit Hamiltonianscited by 1
  10. 2026Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approachcited by 1