# Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation

arXiv: https://arxiv.org/abs/2005.03991 · HTML: https://mlsanity.com/paper/2005.03991/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2020.

## Built on by (most cited first, 30 of 39)

- [Robust Compressed Sensing MRI with Deep Generative Priors](https://mlsanity.com/paper/2108.01368/) (2021)
- [Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers](https://mlsanity.com/paper/2105.08059/) (2021)
- [Early Stopping for Deep Image Prior](https://mlsanity.com/paper/2112.06074/) (2021)
- [Instance-Optimal Compressed Sensing via Posterior Sampling](https://mlsanity.com/paper/2106.11438/) (2021)
- [DINER: Disorder-Invariant Implicit Neural Representation](https://mlsanity.com/paper/2211.07871/) (2022)
- [Flat minima generalize for low-rank matrix recovery](https://mlsanity.com/paper/2203.03756/) (2022)
- [Disorder-invariant Implicit Neural Representation](https://mlsanity.com/paper/2304.00837/) (2023)
- [Interpolating Classifiers Make Few Mistakes](https://mlsanity.com/paper/2101.11815/) (2021)
- [Theoretical Perspectives on Deep Learning Methods in Inverse Problems](https://mlsanity.com/paper/2206.14373/) (2022)
- [Data augmentation for deep learning based accelerated MRI reconstruction with limited data](https://mlsanity.com/paper/2106.14947/) (2021)
- [NCP: Neural Correspondence Prior for Effective Unsupervised Shape Matching](https://mlsanity.com/paper/2301.05839/) (2023)
- [An Educated Warm Start For Deep Image Prior-Based Micro CT Reconstruction](https://mlsanity.com/paper/2111.11926/) (2021)
- [Self-Validation: Early Stopping for Single-Instance Deep Generative Priors](https://mlsanity.com/paper/2110.12271/) (2021)
- [Implicit Neural Networks with Fourier-Feature Inputs for Free-breathing Cardiac MRI Reconstruction](https://mlsanity.com/paper/2305.06822/) (2023)
- [Generalization Guarantees for Neural Architecture Search with Train-Validation Split](https://mlsanity.com/paper/2104.14132/) (2021)
- [A review and experimental evaluation of deep learning methods for MRI reconstruction](https://mlsanity.com/paper/2109.08618/) (2021)
- [Analysis of Deep Image Prior and Exploiting Self-Guidance for Image Reconstruction](https://mlsanity.com/paper/2402.04097/) (2024)
- [Blind Image Deblurring with Unknown Kernel Size and Substantial Noise](https://mlsanity.com/paper/2208.09483/) (2022)
- [Practical Phase Retrieval Using Double Deep Image Priors](https://mlsanity.com/paper/2211.00799/) (2022)
- [Deep Learning for Accelerated and Robust MRI Reconstruction: a Review](https://mlsanity.com/paper/2404.15692/) (2024)
- [Analyzing the Sample Complexity of Self-Supervised Image Reconstruction Methods](https://mlsanity.com/paper/2305.19079/) (2023)
- [Self-Supervised Scalable Deep Compressed Sensing](https://mlsanity.com/paper/2308.13777/) (2023)
- [Inductive Gradient Adjustment For Spectral Bias In Implicit Neural Representations](https://mlsanity.com/paper/2410.13271/) (2024)
- [A Deep Learning Method for Simultaneous Denoising and Missing Wedge Reconstruction in Cryogenic Electron Tomography](https://mlsanity.com/paper/2311.05539/) (2023)
- [RoDyGS: Robust Dynamic Gaussian Splatting for Casual Videos](https://mlsanity.com/paper/2412.03077/) (2024)
- [Validation and Generalizability of Self-Supervised Image Reconstruction Methods for Undersampled MRI](https://mlsanity.com/paper/2201.12535/) (2022)
- [Continuous Representation Methods, Theories, and Applications: An Overview and Perspectives](https://mlsanity.com/paper/2505.15222/) (2025)
- [Self-diffusion for Solving Inverse Problems](https://mlsanity.com/paper/2510.21417/) (2025)
- [Multilinear Kernel Regression and Imputation via Manifold Learning](https://mlsanity.com/paper/2402.03648/) (2024)
- [Non-Convex Compressed Sensing with Training Data](https://mlsanity.com/paper/2101.08310/) (2021)

## Abstract (arXiv)

Un-trained convolutional neural networks have emerged as highly successful tools for image recovery and restoration. They are capable of solving standard inverse problems such as denoising and compressive sensing with excellent results by simply fitting a neural network model to measurements from a single image or signal without the need for any additional training data. For some applications, this critically requires additional regularization in the form of early stopping the optimization. For signal recovery from a few measurements, however, un-trained convolutional networks have an intriguing self-regularizing property: Even though the network can perfectly fit any image, the network recovers a natural image from few measurements when trained with gradient descent until convergence. In this paper, we provide numerical evidence for this property and study it theoretically. We show that---without any further regularization---an un-trained convolutional neural network can approximately reconstruct signals and images that are sufficiently structured, from a near minimal number of random measurements.