# On Optimal Regularization Parameters via Bilevel Learning

arXiv: https://arxiv.org/abs/2305.18394 · HTML: https://mlsanity.com/paper/2305.18394/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 81 references that are in the map.
- First posted on arXiv 28 May 2023.

## Builds on (most cited first, 19 of 19)

- [Input Convex Neural Networks](https://mlsanity.com/paper/1609.07152/) (2016)
- [Approximation Methods for Bilevel Programming](https://mlsanity.com/paper/1802.02246/) (2018)
- [A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applications](https://mlsanity.com/paper/1705.06270/) (2017)
- [Learned convex regularizers for inverse problems](https://mlsanity.com/paper/2008.02839/) (2020)
- [Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation](https://mlsanity.com/paper/2205.03076/) (2022)
- [Learning the Sampling Pattern for MRI](https://mlsanity.com/paper/1906.08754/) (2019)
- [Learning Regularization Parameters of Inverse Problems via Deep Neural Networks](https://mlsanity.com/paper/2104.06594/) (2021)
- [Bilevel parameter learning for higher-order total variation regularisation models](https://mlsanity.com/paper/1508.07243/) (2015)
- [Higher-order total variation approaches and generalisations](https://mlsanity.com/paper/1912.01587/) (2019)
- [Techniques for Gradient Based Bilevel Optimization with Nonsmooth Lower Level Problems](https://mlsanity.com/paper/1602.07080/) (2016)
- [A Bilevel Approach for Parameter Learning in Inverse Problems](https://mlsanity.com/paper/1802.01365/) (2018)
- [Analyzing Inexact Hypergradients for Bilevel Learning](https://mlsanity.com/paper/2301.04764/) (2023)
- [Inexact Derivative-Free Optimization for Bilevel Learning](https://mlsanity.com/paper/2006.12674/) (2020)
- [Automated Parameter Selection for Total Variation Minimization in Image Restoration](https://mlsanity.com/paper/1509.07442/) (2015)
- [Parametric Majorization for Data-Driven Energy Minimization Methods](https://mlsanity.com/paper/1908.06209/) (2019)
- [Dualization and Automatic Distributed Parameter Selection of Total Generalized Variation via Bilevel Optimization](https://mlsanity.com/paper/2002.05614/) (2020)
- [Revisiting loss-specific training of filter-based MRFs for image restoration](https://mlsanity.com/paper/1401.4107/) (2014)
- [The structure of optimal parameters for image restoration problems](https://mlsanity.com/paper/1505.01953/) (2015)
- [Bregman Distances in Inverse Problems and Partial Differential Equation](https://mlsanity.com/paper/1505.05191/) (2015)

## Built on by (most cited first, 6 of 6)

- [An Adaptively Inexact Method for Bilevel Learning Using Primal-Dual Style Differentiation](https://mlsanity.com/paper/2412.06436/) (2024)
- [Bilevel Learning with Inexact Stochastic Gradients](https://mlsanity.com/paper/2412.12049/) (2024)
- [Optimal Regularization Under Uncertainty: Distributional Robustness and Convexity Constraints](https://mlsanity.com/paper/2510.03464/) (2025)
- [Individual Fairness Through Reweighting and Tuning](https://mlsanity.com/paper/2405.01711/) (2024)
- [Multibit Quantized Precoding for MU-mMIMO](https://mlsanity.com/paper/2607.15959/) (2026)
- [The Geometry of Anisotropic Dilation for Optimal Regularization](https://mlsanity.com/paper/2610.09310/) (2026)

## Abstract (arXiv)

Variational regularization is commonly used to solve linear inverse problems, and involves augmenting a data fidelity by a regularizer. The regularizer is used to promote a priori information and is weighted by a regularization parameter. Selection of an appropriate regularization parameter is critical, with various choices leading to very different reconstructions. Classical strategies used to determine a suitable parameter value include the discrepancy principle and the L-curve criterion, and in recent years a supervised machine learning approach called bilevel learning has been employed. Bilevel learning is a powerful framework to determine optimal parameters and involves solving a nested optimization problem. While previous strategies enjoy various theoretical results, the well-posedness of bilevel learning in this setting is still an open question. In particular, a necessary property is positivity of the determined regularization parameter. In this work, we provide a new condition that better characterizes positivity of optimal regularization parameters than the existing theory. Numerical results verify and explore this new condition for both small and high-dimensional problems.