# Iterative Reweighted Least Squares Networks With Convergence Guarantees for Solving Inverse Imaging Problems

arXiv: https://arxiv.org/abs/2308.05745 · HTML: https://mlsanity.com/paper/2308.05745/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 76 references that are in the map.
- First posted on arXiv 10 Aug 2023.

## Builds on (most cited first, 24 of 24)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Model](https://mlsanity.com/paper/1904.00523/) (2019)
- [The Little Engine that Could: Regularization by Denoising (RED)](https://mlsanity.com/paper/1611.02862/) (2016)
- ["Zero-Shot" Super-Resolution using Deep Internal Learning](https://mlsanity.com/paper/1712.06087/) (2017)
- [Deep Unfolding Network for Image Super-Resolution](https://mlsanity.com/paper/2003.10428/) (2020)
- [Blind Super-Resolution Kernel Estimation using an Internal-GAN](https://mlsanity.com/paper/1909.06581/) (2019)
- [Deep Equilibrium Architectures for Inverse Problems in Imaging](https://mlsanity.com/paper/2102.07944/) (2021)
- [Residual Feature Distillation Network for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2009.11551/) (2020)
- [Non-Local Color Image Denoising with Convolutional Neural Networks](https://mlsanity.com/paper/1611.06757/) (2016)
- [DWDN: Deep Wiener Deconvolution Network for Non-Blind Image Deblurring](https://mlsanity.com/paper/2103.09962/) (2021)
- [Deep Image Demosaicking using a Cascade of Convolutional Residual Denoising Networks](https://mlsanity.com/paper/1803.05215/) (2018)
- [Weighted Schatten p-Norm Minimization for Image Denoising and Background Subtraction](https://mlsanity.com/paper/1512.01003/) (2015)
- [Iterative Joint Image Demosaicking and Denoising using a Residual Denoising Network](https://mlsanity.com/paper/1807.06403/) (2018)
- [A Scalable Second Order Method for Ill-Conditioned Matrix Completion from Few Samples](https://mlsanity.com/paper/2106.02119/) (2021)
- [Learning Sparse and Low-Rank Priors for Image Recovery via Iterative Reweighted Least Squares Minimization](https://mlsanity.com/paper/2304.10536/) (2023)
- [Iterative Residual CNNs for Burst Photography Applications](https://mlsanity.com/paper/1811.12197/) (2018)
- [Modeling Realistic Degradations in Non-blind Deconvolution](https://mlsanity.com/paper/1806.01097/) (2018)
- [Bilevel learning of l1-regularizers with closed-form gradients(BLORC)](https://mlsanity.com/paper/2111.10858/) (2021)
- [On the convergence of the IRLS algorithm in Non-Local Patch Regression](https://mlsanity.com/paper/1303.0417/) (2013)

## Built on by (most cited first, 1 of 1)

- [Integrating Reweighted Least Squares with Plug-and-Play Diffusion Priors for Noisy Image Restoration](https://mlsanity.com/paper/2511.06823/) (2025)

## Abstract (arXiv)

In this work we present a novel optimization strategy for image reconstruction tasks under analysis-based image regularization, which promotes sparse and/or low-rank solutions in some learned transform domain. We parameterize such regularizers using potential functions that correspond to weighted extensions of the ℓpp-vector and Spp Schatten-matrix quasi-norms with 0 < p ≤ 1. Our proposed minimization strategy extends the Iteratively Reweighted Least Squares (IRLS) method, typically used for synthesis-based ℓp and Sp norm and analysis-based ℓ1 and nuclear norm regularization. We prove that under mild conditions our minimization algorithm converges linearly to a stationary point, and we provide an upper bound for its convergence rate. Further, to select the parameters of the regularizers that deliver the best results for the problem at hand, we propose to learn them from training data by formulating the supervised learning process as a stochastic bilevel optimization problem. We show that thanks to the convergence guarantees of our proposed minimization strategy, such optimization can be successfully performed with a memory-efficient implicit back-propagation scheme. We implement our learned IRLS variants as recurrent networks and assess their performance on the challenging image reconstruction tasks of non-blind deblurring, super-resolution and demosaicking. The comparisons against other existing learned reconstruction approaches demonstrate that our overall method is very competitive and in many cases outperforms existing unrolled networks, whose number of parameters is orders of magnitude higher than in our case.