mlsanityData to 14 Oct 2026

A statistical perspective on algorithm unrolling models for inverse problems

arXiv 2311.06395 ↗First posted 10 Nov 2023stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We consider inverse problems where the conditional distribution of the observation bf y given the latent variable of interest bf x (also known as the forward model) is known, and we have access to a data set in which multiple instances of bf x and bf y are both observed. In this context, algorithm unrolling has become a very popular approach for designing state-of-the-art deep neural network architectures that effectively exploit the forward model. We analyze the statistical complexity of the gradient descent network (GDN), an algorithm unrolling architecture driven by proximal gradient descent. We show that the unrolling depth needed for the optimal statistical performance of GDNs is of order log(n)/log(varrhon-1), where n is the sample size, and varrhon is the convergence rate of the corresponding gradient descent algorithm. We also show that when the negative log-density of the latent variable bf x has a simple proximal operator, then a GDN unrolled at depth D' can solve the inverse problem at the parametric rate O(D'/√n). Our results thus also suggest that algorithm unrolling models are prone to overfitting as the unrolling depth D' increases. We provide several examples to illustrate these results.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 81 references that are in the map. First posted on arXiv 10 Nov 2023.

Builds on

32 of 81 references.

References · 32

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2016Layer Normalizationcited by 5,669
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  5. 2016Error bounds for approximations with deep ReLU networkscited by 574
  6. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 433
  7. 2019Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processingcited by 421
  8. 2017Learning Deep CNN Denoiser Prior for Image Restorationcited by 408
  9. 2017MoDL: Model Based Deep Learning Architecture for Inverse Problemscited by 387
  10. 2020Deep Learning Techniques for Inverse Problems in Imagingcited by 291
  11. 2018Analyzing Inverse Problems with Invertible Neural Networkscited by 242
  12. 2020Deep Network Approximation for Smooth Functionscited by 164
20 more
  1. 2018Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholdscited by 85
  2. 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
  3. 2017One Network to Solve Them All --- Solving Linear Inverse Problems using Deep Projection Modelscited by 67
  4. 2018Posterior Concentration for Sparse Deep Learningcited by 50
  5. 2014Bayesian linear regression with sparse priorscited by 40
  6. 2022Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activationscited by 29
  7. 2022Statistical exploration of the Manifold Hypothesiscited by 29
  8. 2017Learned Convolutional Sparse Codingcited by 26
  9. 2020Statistical Guarantees for Regularized Neural Networkscited by 25
  10. 2018Approximation and Estimation for High-Dimensional Deep Learning Networkscited by 22
  11. 2018On Multi-Layer Basis Pursuit, Efficient Algorithms and Convolutional Neural Networkscited by 21
  12. 2019Invert to Learn to Invertcited by 18
  13. 2017Recent progress in log-concave density estimationcited by 15
  14. 2018Deep BCD-Net Using Identical Encoding-Decoding CNN Structures for Iterative Image Recoverycited by 11
  15. 2015Local Linear Convergence of ISTA and FISTA on the LASSO Problemcited by 8
  16. 2018On Quasi-Newton Forward--Backward Splitting: Proximal Calculus and Convergencecited by 8
  17. 2019Convergence analysis of Tikhonov regularization for non-linear statistical inverse learning problemscited by 7
  18. 2016Ensuring Rapid Mixing and Low Bias for Asynchronous Gibbs Samplingcited by 5
  19. 2015Learning optimal nonlinearities for iterative thresholding algorithmscited by 4
  20. 2021A fast asynchronous MCMC sampler for sparse Bayesian inferencecited by 3

Built on by

Not cited yet.