# Implicit Regularization of the Deep Inverse Prior Trained with Inertia

arXiv: https://arxiv.org/abs/2506.02986 · HTML: https://mlsanity.com/paper/2506.02986/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 37 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [On Lazy Training in Differentiable Programming](https://mlsanity.com/paper/1812.07956/) (2018)
- [Gradient Descent Provably Optimizes Over-parameterized Neural Networks](https://mlsanity.com/paper/1810.02054/) (2018)
- [Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks](https://mlsanity.com/paper/1901.08584/) (2019)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Deep learning: a statistical viewpoint](https://mlsanity.com/paper/2103.09177/) (2021)
- [Towards moderate overparameterization: global convergence guarantees for training shallow neural networks](https://mlsanity.com/paper/1902.04674/) (2019)
- [Overparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path?](https://mlsanity.com/paper/1812.10004/) (2018)
- [Image Restoration using Total Variation Regularized Deep Image Prior](https://mlsanity.com/paper/1810.12864/) (2018)
- [The troublesome kernel -- On hallucinations, no free lunches and the accuracy-stability trade-off in inverse problems](https://mlsanity.com/paper/2001.01258/) (2020)
- [First-order optimization algorithms via inertial systems with Hessian driven damping](https://mlsanity.com/paper/1907.10536/) (2019)
- [Mathematical Models of Overparameterized Neural Networks](https://mlsanity.com/paper/2012.13982/) (2020)
- [An Inertial Newton Algorithm for Deep Learning](https://mlsanity.com/paper/1905.12278/) (2019)
- [BP-DIP: A Backprojection based Deep Image Prior](https://mlsanity.com/paper/2003.05417/) (2020)
- [Deep Internal Learning: Deep Learning from a Single Input](https://mlsanity.com/paper/2312.07425/) (2023)
- [Convergence and Recovery Guarantees of Unsupervised Neural Networks for Inverse Problems](https://mlsanity.com/paper/2309.12128/) (2023)
- [Recovery Guarantees of Unsupervised Neural Networks for Inverse Problems trained with Gradient Descent](https://mlsanity.com/paper/2403.05395/) (2024)
- [Inertial Methods with Viscous and Hessian driven Damping for Non-Convex Optimization](https://mlsanity.com/paper/2407.12518/) (2024)

## Abstract (arXiv)

Solving inverse problems with neural networks benefits from very few theoretical guarantees when it comes to the recovery guarantees. We provide in this work convergence and recovery guarantees for self-supervised neural networks applied to inverse problems, such as Deep Image/Inverse Prior, and trained with inertia featuring both viscous and geometric Hessian-driven dampings. We study both the continuous-time case, i.e., the trajectory of a dynamical system, and the discrete case leading to an inertial algorithm with an adaptive step-size. We show in the continuous-time case that the network can be trained with an optimal accelerated exponential convergence rate compared to the rate obtained with gradient flow. We also show that training a network with our inertial algorithm enjoys similar recovery guarantees though with a less sharp linear convergence rate.