# Rethinking Deep Image Prior for Denoising

arXiv: https://arxiv.org/abs/2108.12841 · HTML: https://mlsanity.com/paper/2108.12841/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 57 references that are in the map.
- First posted on arXiv 29 Aug 2021.

## Builds on (most cited first, 20 of 20)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [Benchmarking Denoising Algorithms with Real Photographs](https://mlsanity.com/paper/1707.01313/) (2017)
- [High-Quality Self-Supervised Deep Image Denoising](https://mlsanity.com/paper/1901.10277/) (2019)
- [Data Distillation: Towards Omni-Supervised Learning](https://mlsanity.com/paper/1712.04440/) (2017)
- [Noisier2Noise: Learning to Denoise from Unpaired Noisy Data](https://mlsanity.com/paper/1910.11908/) (2019)
- [Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks](https://mlsanity.com/paper/1810.03982/) (2018)
- [Neural Nearest Neighbors Networks](https://mlsanity.com/paper/1810.12575/) (2018)
- [DeepRED: Deep Image Prior Powered by RED](https://mlsanity.com/paper/1903.10176/) (2019)
- [A Bayesian Perspective on the Deep Image Prior](https://mlsanity.com/paper/1904.07457/) (2019)
- [Unsupervised Learning with Stein's Unbiased Risk Estimator](https://mlsanity.com/paper/1805.10531/) (2018)
- [Training Deep Learning Based Denoisers without Ground Truth Data](https://mlsanity.com/paper/1803.01314/) (2018)
- [Degrees of Freedom in Deep Neural Networks](https://mlsanity.com/paper/1603.09260/) (2016)

## Built on by (most cited first, 30 of 38)

- [Early Stopping for Deep Image Prior](https://mlsanity.com/paper/2112.06074/) (2021)
- [GSURE-Based Diffusion Model Training with Corrupted Data](https://mlsanity.com/paper/2305.13128/) (2023)
- [Deep Diffusion Image Prior for Efficient OOD Adaptation in 3D Inverse Problems](https://mlsanity.com/paper/2407.10641/) (2024)
- [Learning to Immunize Images for Tamper Localization and Self-Recovery](https://mlsanity.com/paper/2210.15902/) (2022)
- [NCP: Neural Correspondence Prior for Effective Unsupervised Shape Matching](https://mlsanity.com/paper/2301.05839/) (2023)
- [An Educated Warm Start For Deep Image Prior-Based Micro CT Reconstruction](https://mlsanity.com/paper/2111.11926/) (2021)
- [Self-Validation: Early Stopping for Single-Instance Deep Generative Priors](https://mlsanity.com/paper/2110.12271/) (2021)
- [Equivariance and Invariance Inductive Bias for Learning from Insufficient Data](https://mlsanity.com/paper/2207.12258/) (2022)
- [The Devil is in the Upsampling: Architectural Decisions Made Simpler for Denoising with Deep Image Prior](https://mlsanity.com/paper/2304.11409/) (2023)
- [Analysis of Deep Image Prior and Exploiting Self-Guidance for Image Reconstruction](https://mlsanity.com/paper/2402.04097/) (2024)
- [Deep Internal Learning: Deep Learning from a Single Input](https://mlsanity.com/paper/2312.07425/) (2023)
- [Revisiting Implicit Neural Representations in Low-Level Vision](https://mlsanity.com/paper/2304.10250/) (2023)
- [Patch-Based Diffusion Models Beat Whole-Image Models for Mismatched Distribution Inverse Problems](https://mlsanity.com/paper/2410.11730/) (2024)
- [A Baseline Method for Removing Invisible Image Watermarks using Deep Image Prior](https://mlsanity.com/paper/2502.13998/) (2025)
- [Recurrent Spike-based Image Restoration under General Illumination](https://mlsanity.com/paper/2308.03018/) (2023)
- [Sensitivity Decouple Learning for Image Compression Artifacts Reduction](https://mlsanity.com/paper/2405.09291/) (2024)
- [Linear Combinations of Patches are Unreasonably Effective for Single-Image Denoising](https://mlsanity.com/paper/2212.00422/) (2022)
- [Image Denoising Using Green Channel Prior](https://mlsanity.com/paper/2408.05923/) (2024)
- [YOND: Practical Blind Raw Image Denoising Free from Camera-Specific Data Dependency](https://mlsanity.com/paper/2506.03645/) (2025)
- [Priors in Deep Image Restoration and Enhancement: A Survey](https://mlsanity.com/paper/2206.02070/) (2022)
- [Masked Pre-training Enables Universal Zero-shot Denoiser](https://mlsanity.com/paper/2401.14966/) (2024)
- [CFNet: Conditional Filter Learning with Dynamic Noise Estimation for Real Image Denoising](https://mlsanity.com/paper/2211.14576/) (2022)
- [SVD-DIP: Overcoming the Overfitting Problem in DIP-based CT Reconstruction](https://mlsanity.com/paper/2303.15748/) (2023)
- [Learning Self-Prior for Mesh Inpainting Using Self-Supervised Graph Convolutional Networks](https://mlsanity.com/paper/2305.00635/) (2023)
- [Optimal Eye Surgeon: Finding Image Priors through Sparse Generators at Initialization](https://mlsanity.com/paper/2406.05288/) (2024)
- [Residual Back Projection With Untrained Neural Networks](https://mlsanity.com/paper/2210.14416/) (2022)
- [Image Reconstruction via Deep Image Prior Subspaces](https://mlsanity.com/paper/2302.10279/) (2023)
- [On normalization-equivariance properties of supervised and unsupervised denoising methods: a survey](https://mlsanity.com/paper/2402.15352/) (2024)
- [SDIP: Self-Reinforcement Deep Image Prior Framework for Image Processing](https://mlsanity.com/paper/2404.12142/) (2024)
- [Optimize the Unseen -- Fast NeRF Cleanup with Free Space Prior](https://mlsanity.com/paper/2412.12772/) (2024)

## Abstract (arXiv)

Deep image prior (DIP) serves as a good inductive bias for diverse inverse problems. Among them, denoising is known to be particularly challenging for the DIP due to noise fitting with the requirement of an early stopping. To address the issue, we first analyze the DIP by the notion of effective degrees of freedom (DF) to monitor the optimization progress and propose a principled stopping criterion before fitting to noise without access of a paired ground truth image for Gaussian noise. We also propose the `stochastic temporal ensemble (STE)' method for incorporating techniques to further improve DIP's performance for denoising. We additionally extend our method to Poisson noise. Our empirical validations show that given a single noisy image, our method denoises the image while preserving rich textual details. Further, our approach outperforms prior arts in LPIPS by large margins with comparable PSNR and SSIM on seven different datasets.