# Learning the optimal Tikhonov regularizer for inverse problems

arXiv: https://arxiv.org/abs/2106.06513 · HTML: https://mlsanity.com/paper/2106.06513/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 51 references that are in the map.
- First posted on arXiv 11 Jun 2021.

## Builds on (most cited first, 18 of 18)

- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Learned Primal-dual Reconstruction](https://mlsanity.com/paper/1707.06474/) (2017)
- [Solving ill-posed inverse problems using iterative deep neural networks](https://mlsanity.com/paper/1704.04058/) (2017)
- [Deep Equilibrium Architectures for Inverse Problems in Imaging](https://mlsanity.com/paper/2102.07944/) (2021)
- [Adversarial Regularizers in Inverse Problems](https://mlsanity.com/paper/1805.11572/) (2018)
- [The Random Feature Model for Input-Output Maps between Banach Spaces](https://mlsanity.com/paper/2005.10224/) (2020)
- [NETT: Solving Inverse Problems with Deep Neural Networks](https://mlsanity.com/paper/1803.00092/) (2018)
- [Optimal Rates For Regularization Of Statistical Inverse Learning Problems](https://mlsanity.com/paper/1604.04054/) (2016)
- [Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spaces](https://mlsanity.com/paper/1801.06720/) (2018)
- [Learned convex regularizers for inverse problems](https://mlsanity.com/paper/2008.02839/) (2020)
- [Total Deep Variation for Linear Inverse Problems](https://mlsanity.com/paper/2001.05005/) (2020)
- [Multi-Scale Learned Iterative Reconstruction](https://mlsanity.com/paper/1908.00936/) (2019)
- [Bilevel approaches for learning of variational imaging models](https://mlsanity.com/paper/1505.02120/) (2015)
- [Deep learning architectures for nonlinear operator functions and nonlinear inverse problems](https://mlsanity.com/paper/1912.11090/) (2019)
- [Deep neural networks for inverse problems with pseudodifferential operators: an application to limited-angle tomography](https://mlsanity.com/paper/2006.01620/) (2020)
- [Learning Schatten--von Neumann Operators](https://mlsanity.com/paper/1901.10076/) (2019)
- [Adversarially learned iterative reconstruction for imaging inverse problems](https://mlsanity.com/paper/2103.16151/) (2021)

## Built on by (most cited first, 26 of 26)

- [Convergence Rates for Learning Linear Operators from Noisy Data](https://mlsanity.com/paper/2108.12515/) (2021)
- [Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysis](https://mlsanity.com/paper/2210.13300/) (2022)
- [Learning Regularization Functionals for Inverse Problems: A Comparative Study](https://mlsanity.com/paper/2510.01755/) (2025)
- [Weakly Convex Regularisers for Inverse Problems: Convergence of Critical Points and Primal-Dual Optimisation](https://mlsanity.com/paper/2402.01052/) (2024)
- [Convergent Data-driven Regularizations for CT Reconstruction](https://mlsanity.com/paper/2212.07786/) (2022)
- [Good Things Come in Pairs: Paired Autoencoders for Inverse Problems](https://mlsanity.com/paper/2505.06549/) (2025)
- [RN-SDEs: Limited-Angle CT Reconstruction with Residual Null-Space Diffusion Stochastic Differential Equations](https://mlsanity.com/paper/2409.13930/) (2024)
- [A Generalization Bound for a Family of Implicit Networks](https://mlsanity.com/paper/2410.07427/) (2024)
- [Learning via nonlinear conjugate gradients and depth-varying neural ODEs](https://mlsanity.com/paper/2202.05766/) (2022)
- [Learning Variational Models with Unrolling and Bilevel Optimization](https://mlsanity.com/paper/2209.12651/) (2022)
- [Regularization, early-stopping and dreaming: a Hopfield-like setup to address generalization and overfitting](https://mlsanity.com/paper/2308.01421/) (2023)
- [Learning a Gaussian Mixture for Sparsity Regularization in Inverse Problems](https://mlsanity.com/paper/2401.16612/) (2024)
- [Unsupervised Training of Convex Regularizers using Maximum Likelihood Estimation](https://mlsanity.com/paper/2404.05445/) (2024)
- [The Star Geometry of Critic-Based Regularizer Learning](https://mlsanity.com/paper/2408.16852/) (2024)
- [Learning sparsity-promoting regularizers for linear inverse problems](https://mlsanity.com/paper/2412.16031/) (2024)
- [Learned Regularization for Inverse Problems: Insights from a Spectral Model](https://mlsanity.com/paper/2312.09845/) (2023)
- [Learned reconstruction methods for inverse problems: sample error estimates](https://mlsanity.com/paper/2312.14078/) (2023)
- [Statistical inverse learning problems with random observations](https://mlsanity.com/paper/2312.15341/) (2023)
- [sc-OTGM: Single-Cell Perturbation Modeling by Solving Optimal Mass Transport on the Manifold of Gaussian Mixtures](https://mlsanity.com/paper/2405.03726/) (2024)
- [Optimal Regularization Under Uncertainty: Distributional Robustness and Convexity Constraints](https://mlsanity.com/paper/2510.03464/) (2025)
- [Why the noise model matters: A performance gap in learned regularization](https://mlsanity.com/paper/2510.12521/) (2025)
- [RINS-T: Robust Implicit Neural Solvers for Time Series Linear Inverse Problems](https://mlsanity.com/paper/2510.17396/) (2025)
- [Equivariant Sampling for Improving Diffusion Model-based Image Restoration](https://mlsanity.com/paper/2511.09965/) (2025)
- [On the Sample Complexity of Learning for Blind Inverse Problems](https://mlsanity.com/paper/2512.23405/) (2025)
- [Training Implicit Networks for Image Deblurring using Jacobian-Free Backpropagation](https://mlsanity.com/paper/2402.02065/) (2024)
- [Extrapolating from Regularised Solutions for Solving Ill-Conditioned Linear Systems in Machine Learning](https://mlsanity.com/paper/2606.30328/) (2026)

## Abstract (arXiv)

In this work, we consider the linear inverse problem y=Ax+ε, where Acolon X→ Y is a known linear operator between the separable Hilbert spaces X and Y, x is a random variable in X and ε is a zero-mean random process in Y. This setting covers several inverse problems in imaging including denoising, deblurring, and X-ray tomography. Within the classical framework of regularization, we focus on the case where the regularization functional is not given a priori but learned from data. Our first result is a characterization of the optimal generalized Tikhonov regularizer, with respect to the mean squared error. We find that it is completely independent of the forward operator A and depends only on the mean and covariance of x. Then, we consider the problem of learning the regularizer from a finite training set in two different frameworks: one supervised, based on samples of both x and y, and one unsupervised, based only on samples of x. In both cases, we prove generalization bounds, under some weak assumptions on the distribution of x and ε, including the case of sub-Gaussian variables. Our bounds hold in infinite-dimensional spaces, thereby showing that finer and finer discretizations do not make this learning problem harder. The results are validated through numerical simulations.