mlsanityData to 7 Oct 2026

Learning the optimal Tikhonov regularizer for inverse problems

arXiv 2106.06513 ↗First posted 11 Jun 2021stat.ML cs.LG math.STcited by 26

Not on the map. Why?

Abstract (arXiv)

In this work, we consider the linear inverse problem y=Ax+ε, where Acolon X→ Y is a known linear operator between the separable Hilbert spaces X and Y, x is a random variable in X and ε is a zero-mean random process in Y. This setting covers several inverse problems in imaging including denoising, deblurring, and X-ray tomography. Within the classical framework of regularization, we focus on the case where the regularization functional is not given a priori but learned from data. Our first result is a characterization of the optimal generalized Tikhonov regularizer, with respect to the mean squared error. We find that it is completely independent of the forward operator A and depends only on the mean and covariance of x. Then, we consider the problem of learning the regularizer from a finite training set in two different frameworks: one supervised, based on samples of both x and y, and one unsupervised, based only on samples of x. In both cases, we prove generalization bounds, under some weak assumptions on the distribution of x and ε, including the case of sub-Gaussian variables. Our bounds hold in infinite-dimensional spaces, thereby showing that finer and finer discretizations do not make this learning problem harder. The results are validated through numerical simulations.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 18 of its 51 references that are in the map. First posted on arXiv 11 Jun 2021.

Builds on

18 of 51 references.

References · 18

  1. 2017Learning a Variational Network for Reconstruction of Accelerated MRI Datacited by 340
  2. 2020Deep Learning Techniques for Inverse Problems in Imagingcited by 290
  3. 2017Learned Primal-dual Reconstructioncited by 251
  4. 2017Solving ill-posed inverse problems using iterative deep neural networkscited by 145
  5. 2021Deep Equilibrium Architectures for Inverse Problems in Imagingcited by 126
  6. 2018Adversarial Regularizers in Inverse Problemscited by 102
  7. 2020The Random Feature Model for Input-Output Maps between Banach Spacescited by 90
  8. 2018NETT: Solving Inverse Problems with Deep Neural Networkscited by 89
  9. 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
  10. 2018Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spacescited by 59
  11. 2020Learned convex regularizers for inverse problemscited by 54
  12. 2020Total Deep Variation for Linear Inverse Problemscited by 53
6 more
  1. 2019Multi-Scale Learned Iterative Reconstructioncited by 18
  2. 2015Bilevel approaches for learning of variational imaging modelscited by 17
  3. 2019Deep learning architectures for nonlinear operator functions and nonlinear inverse problemscited by 17
  4. 2020Deep neural networks for inverse problems with pseudodifferential operators: an application to limited-angle tomographycited by 15
  5. 2019Learning Schatten--von Neumann Operatorscited by 7
  6. 2021Adversarially learned iterative reconstruction for imaging inverse problemscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 4; 2023: 4; 2024: 9; 2025: 7; 2026: 1.

26 papers.

Sort:

By area:

  1. 2021Convergence Rates for Learning Linear Operators from Noisy Datacited by 55
  2. 2022Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysiscited by 22
  3. 2024Weakly Convex Regularisers for Inverse Problems: Convergence of Critical Points and Primal-Dual Optimisationcited by 12
  4. 2025Learning Regularization Functionals for Inverse Problems: A Comparative Studycited by 12
  5. 2022Convergent Data-driven Regularizations for CT Reconstructioncited by 5
  6. 2025Good Things Come in Pairs: Paired Autoencoders for Inverse Problemscited by 5
  7. 2024A Generalization Bound for a Family of Implicit Networkscited by 4
  8. 2022Learning via nonlinear conjugate gradients and depth-varying neural ODEscited by 3
  9. 2022Learning Variational Models with Unrolling and Bilevel Optimizationcited by 3
  10. 2023Regularization, early-stopping and dreaming: a Hopfield-like setup to address generalization and overfittingcited by 3
  11. 2024Learning a Gaussian Mixture for Sparsity Regularization in Inverse Problemscited by 3
  12. 2024Unsupervised Training of Convex Regularizers using Maximum Likelihood Estimationcited by 3
14 more
  1. 2024The Star Geometry of Critic-Based Regularizer Learningcited by 3
  2. 2024RN-SDEs: Limited-Angle CT Reconstruction with Residual Null-Space Diffusion Stochastic Differential Equationscited by 3
  3. 2024Learning sparsity-promoting regularizers for linear inverse problemscited by 3
  4. 2023Learned Regularization for Inverse Problems: Insights from a Spectral Modelcited by 2
  5. 2023Learned reconstruction methods for inverse problems: sample error estimatescited by 1
  6. 2023Statistical inverse learning problems with random observationscited by 1
  7. 2024sc-OTGM: Single-Cell Perturbation Modeling by Solving Optimal Mass Transport on the Manifold of Gaussian Mixturescited by 1
  8. 2025Why the noise model matters: A performance gap in learned regularizationcited by 1
  9. 2025RINS-T: Robust Implicit Neural Solvers for Time Series Linear Inverse Problemscited by 1
  10. 2025Equivariant Sampling for Improving Diffusion Model-based Image Restorationcited by 1
  11. 2025On the Sample Complexity of Learning for Blind Inverse Problemscited by 1
  12. 2024Training Implicit Networks for Image Deblurring using Jacobian-Free Backpropagationcited by 0
  13. 2025Optimal Regularization Under Uncertainty: Distributional Robustness and Convexity Constraintscited by 0
  14. 2026Extrapolating from Regularised Solutions for Solving Ill-Conditioned Linear Systems in Machine Learningcited by 0