# Learning Regularization Parameters of Inverse Problems via Deep Neural Networks

arXiv: https://arxiv.org/abs/2104.06594 · HTML: https://mlsanity.com/paper/2104.06594/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 89 references that are in the map.
- First posted on arXiv 14 Apr 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [CNN Features off-the-shelf: an Astounding Baseline for Recognition](https://mlsanity.com/paper/1403.6382/) (2014)
- [Learned Primal-dual Reconstruction](https://mlsanity.com/paper/1707.06474/) (2017)
- [NETT: Solving Inverse Problems with Deep Neural Networks](https://mlsanity.com/paper/1803.00092/) (2018)
- [A Review of Convolutional Neural Networks for Inverse Problems in Imaging](https://mlsanity.com/paper/1710.04011/) (2017)
- [Regularization by architecture: A deep prior approach for inverse problems](https://mlsanity.com/paper/1812.03889/) (2018)
- [Bilevel approaches for learning of variational imaging models](https://mlsanity.com/paper/1505.02120/) (2015)
- [Bilevel parameter learning for higher-order total variation regularisation models](https://mlsanity.com/paper/1508.07243/) (2015)
- [Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection](https://mlsanity.com/paper/2007.13171/) (2020)
- [Bilevel Optimization, Deep Learning and Fractional Laplacian Regularization with Applications in Tomography](https://mlsanity.com/paper/1907.09605/) (2019)
- [Automated Parameter Selection for Total Variation Minimization in Image Restoration](https://mlsanity.com/paper/1509.07442/) (2015)
- [Learning Priors for Adversarial Autoencoders](https://mlsanity.com/paper/1909.04443/) (2019)

## Built on by (most cited first, 19 of 19)

- [DRIP: Deep Regularizers for Inverse Problems](https://mlsanity.com/paper/2304.00015/) (2023)
- [Transformer learns the cross-task prior and regularization for in-context learning](https://mlsanity.com/paper/2505.12138/) (2025)
- [On Optimal Regularization Parameters via Bilevel Learning](https://mlsanity.com/paper/2305.18394/) (2023)
- [Paired Autoencoders for Likelihood-free Estimation in Inverse Problems](https://mlsanity.com/paper/2405.13220/) (2024)
- [Good Things Come in Pairs: Paired Autoencoders for Inverse Problems](https://mlsanity.com/paper/2505.06549/) (2025)
- [Learning Causal Graphs at Scale: A Foundation Model Approach](https://mlsanity.com/paper/2506.18285/) (2025)
- [Deep unrolling for learning optimal spatially varying regularisation parameters for Total Generalised Variation](https://mlsanity.com/paper/2502.16532/) (2025)
- [Goal-oriented Uncertainty Quantification for Inverse Problems via Variational Encoder-Decoder Networks](https://mlsanity.com/paper/2304.08324/) (2023)
- [Adaptive Block Sparse Regularization under Arbitrary Linear Transform](https://mlsanity.com/paper/2401.15292/) (2024)
- [DeepMpMRI: Tensor-decomposition Regularized Learning for Fast and High-Fidelity Multi-Parametric Microstructural MR Imaging](https://mlsanity.com/paper/2405.03159/) (2024)
- [Learning end-to-end inversion of circular Radon transforms in the partial radial setup](https://mlsanity.com/paper/2308.14144/) (2023)
- [Training Implicit Networks for Image Deblurring using Jacobian-Free Backpropagation](https://mlsanity.com/paper/2402.02065/) (2024)
- [Bayesian Model Parameter Learning in Linear Inverse Problems: Application in EEG Focal Source Imaging](https://mlsanity.com/paper/2501.13109/) (2025)
- [Optimization Landscapes Learned: Proxy Networks Boost Convergence in Physics-based Inverse Problems](https://mlsanity.com/paper/2501.16573/) (2025)
- [Deep regularization networks for inverse problems with noisy operators](https://mlsanity.com/paper/2506.07008/) (2025)
- [Optimal Linear Baseline Models for Scientific Machine Learning](https://mlsanity.com/paper/2508.05831/) (2025)
- [Latent Space Inference via Paired Autoencoders](https://mlsanity.com/paper/2601.11397/) (2026)
- [Oracle Supervision Transfers for Hyperparameter Prediction in Model-Based Image Denoising](https://mlsanity.com/paper/2605.20479/) (2026)
- [Learning spatially varying regularisation parameters of low regularity for image reconstruction](https://mlsanity.com/paper/2608.25127/) (2026)

## Abstract (arXiv)

In this work, we describe a new approach that uses deep neural networks (DNN) to obtain regularization parameters for solving inverse problems. We consider a supervised learning approach, where a network is trained to approximate the mapping from observation data to regularization parameters. Once the network is trained, regularization parameters for newly obtained data can be computed by efficient forward propagation of the DNN. We show that a wide variety of regularization functionals, forward models, and noise models may be considered. The network-obtained regularization parameters can be computed more efficiently and may even lead to more accurate solutions compared to existing regularization parameter selection methods. We emphasize that the key advantage of using DNNs for learning regularization parameters, compared to previous works on learning via optimal experimental design or empirical Bayes risk minimization, is greater generalizability. That is, rather than computing one set of parameters that is optimal with respect to one particular design objective, DNN-computed regularization parameters are tailored to the specific features or properties of the newly observed data. Thus, our approach may better handle cases where the observation is not a close representation of the training set. Furthermore, we avoid the need for expensive and challenging bilevel optimization methods as utilized in other existing training approaches. Numerical results demonstrate the potential of using DNNs to learn regularization parameters.