# Convergence and Recovery Guarantees of Unsupervised Neural Networks for Inverse Problems

arXiv: https://arxiv.org/abs/2309.12128 · HTML: https://mlsanity.com/paper/2309.12128/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 61 references that are in the map.
- First posted on arXiv 21 Sep 2023.

## Builds on (most cited first, 30 of 33)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [On Lazy Training in Differentiable Programming](https://mlsanity.com/paper/1812.07956/) (2018)
- [Gradient Descent Provably Optimizes Over-parameterized Neural Networks](https://mlsanity.com/paper/1810.02054/) (2018)
- [Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks](https://mlsanity.com/paper/1901.08584/) (2019)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [An Introduction to Matrix Concentration Inequalities](https://mlsanity.com/paper/1501.01571/) (2015)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Deep learning: a statistical viewpoint](https://mlsanity.com/paper/2103.09177/) (2021)
- [On instabilities of deep learning in image reconstruction - Does AI come at a cost?](https://mlsanity.com/paper/1902.05300/) (2019)
- [Towards moderate overparameterization: global convergence guarantees for training shallow neural networks](https://mlsanity.com/paper/1902.04674/) (2019)
- [NETT: Solving Inverse Problems with Deep Neural Networks](https://mlsanity.com/paper/1803.00092/) (2018)
- [Overparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path?](https://mlsanity.com/paper/1812.10004/) (2018)
- [DeepRED: Deep Image Prior Powered by RED](https://mlsanity.com/paper/1903.10176/) (2019)
- [DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data](https://mlsanity.com/paper/1706.02071/) (2017)
- [Image Restoration using Total Variation Regularized Deep Image Prior](https://mlsanity.com/paper/1810.12864/) (2018)
- [Learned convex regularizers for inverse problems](https://mlsanity.com/paper/2008.02839/) (2020)
- [Learned reconstruction methods with convergence guarantees](https://mlsanity.com/paper/2206.05431/) (2022)
- [From error bounds to the complexity of first-order descent methods for convex functions](https://mlsanity.com/paper/1510.08234/) (2015)
- [Can Push-forward Generative Models Fit Multimodal Distributions?](https://mlsanity.com/paper/2206.14476/) (2022)
- [The troublesome kernel -- On hallucinations, no free lunches and the accuracy-stability trade-off in inverse problems](https://mlsanity.com/paper/2001.01258/) (2020)
- [On Measuring and Controlling the Spectral Bias of the Deep Image Prior](https://mlsanity.com/paper/2107.01125/) (2021)
- [Algorithmic Guarantees for Inverse Imaging with Untrained Network Priors](https://mlsanity.com/paper/1906.08763/) (2019)
- [Stabilizing Invertible Neural Networks Using Mixture Models](https://mlsanity.com/paper/2009.02994/) (2020)
- [Disconnected Manifold Learning for Generative Adversarial Networks](https://mlsanity.com/paper/1806.00880/) (2018)
- [Learning disconnected manifolds: a no GANs land](https://mlsanity.com/paper/2006.04596/) (2020)
- [Recovery Analysis for Plug-and-Play Priors using the Restricted Eigenvalue Condition](https://mlsanity.com/paper/2106.03668/) (2021)
- [Learning local regularization for variational image restoration](https://mlsanity.com/paper/2102.06155/) (2021)
- [Mathematical Models of Overparameterized Neural Networks](https://mlsanity.com/paper/2012.13982/) (2020)

## Built on by (most cited first, 3 of 3)

- [Recovery Guarantees of Unsupervised Neural Networks for Inverse Problems trained with Gradient Descent](https://mlsanity.com/paper/2403.05395/) (2024)
- [Implicit Regularization of the Deep Inverse Prior Trained with Inertia](https://mlsanity.com/paper/2506.02986/) (2025)
- [CogGen: Cognitive-Load-Inspired Fully Unsupervised Deep Generative Modeling for Compressively Sampled MRI Reconstruction](https://mlsanity.com/paper/2603.04438/) (2026)

## Abstract (arXiv)

Neural networks have become a prominent approach to solve inverse problems in recent years. While a plethora of such methods was developed to solve inverse problems empirically, we are still lacking clear theoretical guarantees for these methods. On the other hand, many works proved convergence to optimal solutions of neural networks in a more general setting using overparametrization as a way to control the Neural Tangent Kernel. In this work we investigate how to bridge these two worlds and we provide deterministic convergence and recovery guarantees for the class of unsupervised feedforward multilayer neural networks trained to solve inverse problems. We also derive overparametrization bounds under which a two-layers Deep Inverse Prior network with smooth activation function will benefit from our guarantees.