# Understanding Untrained Deep Models for Inverse Problems: Algorithms and Theory

arXiv: https://arxiv.org/abs/2502.18612 · HTML: https://mlsanity.com/paper/2502.18612/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2025.

## Built on by (most cited first, 5 of 5)

- [FMPlug: Plug-In Foundation Flow-Matching Priors for Inverse Problems](https://mlsanity.com/paper/2508.00721/) (2025)
- [Compressive Imaging Reconstruction via Tensor Decomposed Multi-Resolution Grid Encoding](https://mlsanity.com/paper/2507.07707/) (2025)
- [On the Dataless Training of Neural Networks](https://mlsanity.com/paper/2510.25962/) (2025)
- [Tada-DIP: Input-adaptive Deep Image Prior for One-shot 3D Image Reconstruction](https://mlsanity.com/paper/2512.03962/) (2025)
- [Preventing Overfitting in Deep Image Prior for Hyperspectral Image Denoising](https://mlsanity.com/paper/2604.08272/) (2026)

## Abstract (arXiv)

In recent years, deep learning methods have been extensively developed for inverse imaging problems (IIPs), encompassing supervised, self-supervised, and generative approaches. Most of these methods require large amounts of labeled or unlabeled training data to learn effective models. However, in many practical applications, such as medical image reconstruction, extensive training datasets are often unavailable or limited. A significant milestone in addressing this challenge came in 2018 with the work of Ulyanov et al., which introduced the Deep Image Prior (DIP)--the first training-data-free neural network method for IIPs. Unlike conventional deep learning approaches, DIP requires only a convolutional neural network, the noisy measurements, and a forward operator. By leveraging the implicit regularization of deep networks initialized with random noise, DIP can learn and restore image structures without relying on external datasets. However, a well-known limitation of DIP is its susceptibility to overfitting, primarily due to the over-parameterization of the network. In this tutorial paper, we provide a comprehensive review of DIP, including a theoretical analysis of its training dynamics. We also categorize and discuss recent advancements in DIP-based methods aimed at mitigating overfitting, including techniques such as regularization, network re-parameterization, and early stopping. Furthermore, we discuss approaches that combine DIP with pre-trained neural networks, present empirical comparison results against data-centric methods, and highlight open research questions and future directions.