# Early Stopping for Deep Image Prior

arXiv: https://arxiv.org/abs/2112.06074 · HTML: https://mlsanity.com/paper/2112.06074/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 74 references that are in the map.
- First posted on arXiv 11 Dec 2021.

## Builds on (most cited first, 30 of 42)

- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [On Lazy Training in Differentiable Programming](https://mlsanity.com/paper/1812.07956/) (2018)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art](https://mlsanity.com/paper/1704.05519/) (2017)
- [On the linearity of large non-linear models: when and why the tangent kernel is constant](https://mlsanity.com/paper/2010.01092/) (2020)
- [Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks](https://mlsanity.com/paper/1810.03982/) (2018)
- ["Double-DIP": Unsupervised Image Decomposition via Coupled Deep-Image-Priors](https://mlsanity.com/paper/1812.00467/) (2018)
- [Neural Blind Deconvolution Using Deep Priors](https://mlsanity.com/paper/1908.02197/) (2019)
- [Deep Geometric Prior for Surface Reconstruction](https://mlsanity.com/paper/1811.10943/) (2018)
- [Rethinking Bias-Variance Trade-off for Generalization of Neural Networks](https://mlsanity.com/paper/2002.11328/) (2020)
- [DeepRED: Deep Image Prior Powered by RED](https://mlsanity.com/paper/1903.10176/) (2019)
- [Implicit Regularization for Optimal Sparse Recovery](https://mlsanity.com/paper/1909.05122/) (2019)
- [Real-world Noisy Image Denoising: A New Benchmark](https://mlsanity.com/paper/1804.02603/) (2018)
- [Phase Retrieval Under a Generative Prior](https://mlsanity.com/paper/1807.04261/) (2018)
- [Time-Dependent Deep Image Prior for Dynamic MRI](https://mlsanity.com/paper/1910.01684/) (2019)
- [Image Restoration using Total Variation Regularized Deep Image Prior](https://mlsanity.com/paper/1810.12864/) (2018)
- [Explore Image Deblurring via Blur Kernel Space](https://mlsanity.com/paper/2104.00317/) (2021)
- [Computed Tomography Reconstruction Using Deep Image Prior and Learned Reconstruction Methods](https://mlsanity.com/paper/2003.04989/) (2020)
- [A Bayesian Perspective on the Deep Image Prior](https://mlsanity.com/paper/1904.07457/) (2019)
- [NTIRE 2020 Challenge on Real Image Denoising: Dataset, Methods and Results](https://mlsanity.com/paper/2005.04117/) (2020)
- [Unsupervised Learning with Stein's Unbiased Risk Estimator](https://mlsanity.com/paper/1805.10531/) (2018)
- [Compressed Sensing with Deep Image Prior and Learned Regularization](https://mlsanity.com/paper/1806.06438/) (2018)
- [Blind Image Deconvolution using Deep Generative Priors](https://mlsanity.com/paper/1802.04073/) (2018)
- [Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators](https://mlsanity.com/paper/1910.14634/) (2019)

## Built on by (most cited first, 30 of 44)

- [DMPlug: A Plug-in Method for Solving Inverse Problems with Diffusion Models](https://mlsanity.com/paper/2405.16749/) (2024)
- [Algorithmic Regularization in Model-free Overparametrized Asymmetric Matrix Factorization](https://mlsanity.com/paper/2203.02839/) (2022)
- [The Devil is in the Upsampling: Architectural Decisions Made Simpler for Denoising with Deep Image Prior](https://mlsanity.com/paper/2304.11409/) (2023)
- [Uncertainty Estimation for Computed Tomography with a Linearised Deep Image Prior](https://mlsanity.com/paper/2203.00479/) (2022)
- [Blind Image Deblurring with Unknown Kernel Size and Substantial Noise](https://mlsanity.com/paper/2208.09483/) (2022)
- [Practical Phase Retrieval Using Double Deep Image Priors](https://mlsanity.com/paper/2211.00799/) (2022)
- [Deep Internal Learning: Deep Learning from a Single Input](https://mlsanity.com/paper/2312.07425/) (2023)
- [A Validation Approach to Over-parameterized Matrix and Image Recovery](https://mlsanity.com/paper/2209.10675/) (2022)
- [NCVX: A General-Purpose Optimization Solver for Constrained Machine and Deep Learning](https://mlsanity.com/paper/2210.00973/) (2022)
- [A Baseline Method for Removing Invisible Image Watermarks using Deep Image Prior](https://mlsanity.com/paper/2502.13998/) (2025)
- [Detecting Reddit Users with Depression Using a Hybrid Neural Network SBERT-CNN](https://mlsanity.com/paper/2302.02759/) (2023)
- [Interpretable Deep Learning Methods for Multiview Learning](https://mlsanity.com/paper/2302.07930/) (2023)
- [PIP: Positional-encoding Image Prior](https://mlsanity.com/paper/2211.14298/) (2022)
- [Joint Demosaicing and Denoising with Double Deep Image Priors](https://mlsanity.com/paper/2309.09426/) (2023)
- [Towards Architecture-Agnostic Untrained Network Priors for Image Reconstruction with Frequency Regularization](https://mlsanity.com/paper/2312.09988/) (2023)
- [UGoDIT: Unsupervised Group Deep Image Prior Via Transferable Weights](https://mlsanity.com/paper/2505.11720/) (2025)
- [FMPlug: Plug-In Foundation Flow-Matching Priors for Inverse Problems](https://mlsanity.com/paper/2508.00721/) (2025)
- [CNN-based Euler's Elastica Inpainting with Deep Energy and Deep Image Prior](https://mlsanity.com/paper/2207.07921/) (2022)
- [SVD-DIP: Overcoming the Overfitting Problem in DIP-based CT Reconstruction](https://mlsanity.com/paper/2303.15748/) (2023)
- [Boosting of Implicit Neural Representation-based Image Denoiser](https://mlsanity.com/paper/2401.01548/) (2024)
- [Optimal Eye Surgeon: Finding Image Priors through Sparse Generators at Initialization](https://mlsanity.com/paper/2406.05288/) (2024)
- [An unsupervised method for MRI recovery: Deep image prior with structured sparsity](https://mlsanity.com/paper/2501.01482/) (2025)
- [A Stability Benchmark of Generative Regularizers for Inverse Problems](https://mlsanity.com/paper/2605.10076/) (2026)
- [Image Reconstruction via Deep Image Prior Subspaces](https://mlsanity.com/paper/2302.10279/) (2023)
- [MCDIP-ADMM: Overcoming Overfitting in DIP-based CT reconstruction](https://mlsanity.com/paper/2304.03895/) (2023)
- [Self-Supervised Hyperspectral Inpainting with the Optimisation inspired Deep Neural Network Prior](https://mlsanity.com/paper/2306.07308/) (2023)
- [Self-supervised Deep Hyperspectral Inpainting with the Sparsity and Low-Rank Considerations](https://mlsanity.com/paper/2306.08128/) (2023)
- [Untrained Neural Nets for Snapshot Compressive Imaging: Theory and Algorithms](https://mlsanity.com/paper/2406.03694/) (2024)
- [Self-supervised Deep Hyperspectral Inpainting with the Plug and Play and Deep Image Prior Models](https://mlsanity.com/paper/2501.08195/) (2025)
- [Data-Efficient Limited-Angle CT Using Deep Priors and Regularization](https://mlsanity.com/paper/2502.12293/) (2025)

## Abstract (arXiv)

Deep image prior (DIP) and its variants have showed remarkable potential for solving inverse problems in computer vision, without any extra training data. Practical DIP models are often substantially overparameterized. During the fitting process, these models learn mostly the desired visual content first, and then pick up the potential modeling and observational noise, i.e., overfitting. Thus, the practicality of DIP often depends critically on good early stopping (ES) that captures the transition period. In this regard, the majority of DIP works for vision tasks only demonstrates the potential of the models -- reporting the peak performance against the ground truth, but provides no clue about how to operationally obtain near-peak performance without access to the groundtruth. In this paper, we set to break this practicality barrier of DIP, and propose an efficient ES strategy, which consistently detects near-peak performance across several vision tasks and DIP variants. Based on a simple measure of dispersion of consecutive DIP reconstructions, our ES method not only outpaces the existing ones -- which only work in very narrow domains, but also remains effective when combined with a number of methods that try to mitigate the overfitting. The code is available at https://github.com/sun-umn/Early_Stopping_for_DIP.