# Convergence Guarantees of Overparametrized Wide Deep Inverse Prior

arXiv: https://arxiv.org/abs/2303.11265 · HTML: https://mlsanity.com/paper/2303.11265/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 19 references that are in the map.
- First posted on arXiv 20 Mar 2023.

## Builds on (most cited first, 18 of 18)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [On Lazy Training in Differentiable Programming](https://mlsanity.com/paper/1812.07956/) (2018)
- [Gradient Descent Provably Optimizes Over-parameterized Neural Networks](https://mlsanity.com/paper/1810.02054/) (2018)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [An Introduction to Matrix Concentration Inequalities](https://mlsanity.com/paper/1501.01571/) (2015)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Deep learning: a statistical viewpoint](https://mlsanity.com/paper/2103.09177/) (2021)
- [Towards moderate overparameterization: global convergence guarantees for training shallow neural networks](https://mlsanity.com/paper/1902.04674/) (2019)
- [Overparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path?](https://mlsanity.com/paper/1812.10004/) (2018)
- [DeepRED: Deep Image Prior Powered by RED](https://mlsanity.com/paper/1903.10176/) (2019)
- [Image Restoration using Total Variation Regularized Deep Image Prior](https://mlsanity.com/paper/1810.12864/) (2018)
- [Learned reconstruction methods with convergence guarantees](https://mlsanity.com/paper/2206.05431/) (2022)
- [On Measuring and Controlling the Spectral Bias of the Deep Image Prior](https://mlsanity.com/paper/2107.01125/) (2021)
- [Learning local regularization for variational image restoration](https://mlsanity.com/paper/2102.06155/) (2021)
- [Mathematical Models of Overparameterized Neural Networks](https://mlsanity.com/paper/2012.13982/) (2020)
- [BP-DIP: A Backprojection based Deep Image Prior](https://mlsanity.com/paper/2003.05417/) (2020)

## Built on by (most cited first, 2 of 2)

- [Convergence and Recovery Guarantees of Unsupervised Neural Networks for Inverse Problems](https://mlsanity.com/paper/2309.12128/) (2023)
- [CogGen: Cognitive-Load-Inspired Fully Unsupervised Deep Generative Modeling for Compressively Sampled MRI Reconstruction](https://mlsanity.com/paper/2603.04438/) (2026)

## Abstract (arXiv)

Neural networks have become a prominent approach to solve inverse problems in recent years. Amongst the different existing methods, the Deep Image/Inverse Priors (DIPs) technique is an unsupervised approach that optimizes a highly overparametrized neural network to transform a random input into an object whose image under the forward model matches the observation. However, the level of overparametrization necessary for such methods remains an open problem. In this work, we aim to investigate this question for a two-layers neural network with a smooth activation function. We provide overparametrization bounds under which such network trained via continuous-time gradient descent will converge exponentially fast with high probability which allows to derive recovery prediction bounds. This work is thus a first step towards a theoretical understanding of overparametrized DIP networks, and more broadly it participates to the theoretical understanding of neural networks in inverse problem settings.