# Bilevel methods for image reconstruction

arXiv: https://arxiv.org/abs/2109.09610 · HTML: https://mlsanity.com/paper/2109.09610/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2021.

## Built on by (most cited first, 20 of 20)

- [On Penalty-based Bilevel Gradient Descent Method](https://mlsanity.com/paper/2302.05185/) (2023)
- [First-order penalty methods for bilevel optimization](https://mlsanity.com/paper/2301.01716/) (2023)
- [An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learning](https://mlsanity.com/paper/2308.00788/) (2023)
- [Projection-Free Algorithm for Stochastic Bi-level Optimization](https://mlsanity.com/paper/2110.11721/) (2021)
- [Analyzing Inexact Hypergradients for Bilevel Learning](https://mlsanity.com/paper/2301.04764/) (2023)
- [A Generalized Alternating Method for Bilevel Learning under the Polyak-Lojasiewicz Condition](https://mlsanity.com/paper/2306.02422/) (2023)
- [Joint Unsupervised and Supervised Training for Automatic Speech Recognition via Bilevel Optimization](https://mlsanity.com/paper/2401.06980/) (2024)
- [Unrolled Variational Bayesian Algorithm for Image Blind Deconvolution](https://mlsanity.com/paper/2110.07202/) (2021)
- [Learning Sparsity-Promoting Regularizers using Bilevel Optimization](https://mlsanity.com/paper/2207.08939/) (2022)
- [An adaptively inexact first-order method for bilevel optimization with application to hyperparameter learning](https://mlsanity.com/paper/2308.10098/) (2023)
- [Whiteness-based bilevel learning of regularization parameters in imaging](https://mlsanity.com/paper/2403.07026/) (2024)
- [A primal-dual algorithm for image reconstruction with input-convex neural network regularizers](https://mlsanity.com/paper/2410.12441/) (2024)
- [Solving bilevel optimization via sequential minimax optimization](https://mlsanity.com/paper/2511.07398/) (2025)
- [An Adaptively Inexact Method for Bilevel Learning Using Primal-Dual Style Differentiation](https://mlsanity.com/paper/2412.06436/) (2024)
- [Bilevel Learning with Inexact Stochastic Gradients](https://mlsanity.com/paper/2412.12049/) (2024)
- [Restarted contractive operators to learn at equilibrium](https://mlsanity.com/paper/2506.13239/) (2025)
- [Adaptive Local Neighborhood-based Neural Networks for MR Image Reconstruction from Undersampled Data](https://mlsanity.com/paper/2206.00775/) (2022)
- [Heterogeneous Self-Supervised Acoustic Pre-Training with Local Constraints](https://mlsanity.com/paper/2508.19990/) (2025)
- [Majorization-Minimization Networks for Inverse Problems: An Application to EEG Imaging](https://mlsanity.com/paper/2602.03855/) (2026)
- [Learning spatially varying regularisation parameters of low regularity for image reconstruction](https://mlsanity.com/paper/2608.25127/) (2026)

## Abstract (arXiv)

This review discusses methods for learning parameters for image reconstruction problems using bilevel formulations. Image reconstruction typically involves optimizing a cost function to recover a vector of unknown variables that agrees with collected measurements and prior assumptions. State-of-the-art image reconstruction methods learn these prior assumptions from training data using various machine learning techniques, such as bilevel methods. One can view the bilevel problem as formalizing hyperparameter optimization, as bridging machine learning and cost function based optimization methods, or as a method to learn variables best suited to a specific task. More formally, bilevel problems attempt to minimize an upper-level loss function, where variables in the upper-level loss function are themselves minimizers of a lower-level cost function. This review contains a running example problem of learning tuning parameters and the coefficients for sparsifying filters used in a regularizer. Such filters generalize the popular total variation regularization method, and learned filters are closely related to convolutional neural networks approaches that are rapidly gaining in popularity. Here, the lower-level problem is to reconstruct an image using a regularizer with learned sparsifying filters; the corresponding upper-level optimization problem involves a measure of reconstructed image quality based on training data. This review discusses multiple perspectives to motivate the use of bilevel methods and to make them more easily accessible to different audiences. We then turn to ways to optimize the bilevel problem, providing pros and cons of the variety of proposed approaches. Finally we overview bilevel applications in image reconstruction.