# Iterative regularization for low complexity regularizers

arXiv: https://arxiv.org/abs/2202.00420 · HTML: https://mlsanity.com/paper/2202.00420/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 105 references that are in the map.
- First posted on arXiv 1 Feb 2022.

## Builds on (most cited first, 21 of 21)

- [Implicit Regularization in Matrix Factorization](https://mlsanity.com/paper/1705.09280/) (2017)
- [Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss](https://mlsanity.com/paper/2002.04486/) (2020)
- [Characterizing Implicit Bias in Terms of Optimization Geometry](https://mlsanity.com/paper/1802.08246/) (2018)
- [Linearized two-layers neural networks in high dimension](https://mlsanity.com/paper/1904.12191/) (2019)
- [Early stopping and non-parametric regression: An optimal data-dependent stopping rule](https://mlsanity.com/paper/1306.3574/) (2013)
- [Implicit Regularization for Optimal Sparse Recovery](https://mlsanity.com/paper/1909.05122/) (2019)
- [Early stopping for kernel boosting algorithms: A general analysis with localized complexities](https://mlsanity.com/paper/1707.01543/) (2017)
- [The Statistical Complexity of Early-Stopped Mirror Descent](https://mlsanity.com/paper/2002.00189/) (2020)
- [Implicit Regularization of Accelerated Methods in Hilbert Spaces](https://mlsanity.com/paper/1905.13000/) (2019)
- [Sparse Recovery via Differential Inclusions](https://mlsanity.com/paper/1406.7728/) (2014)
- [Nonsmoothness in Machine Learning: specific structure, proximal identification, and applications](https://mlsanity.com/paper/2010.00848/) (2020)
- [Safe squeezing for antisparse coding](https://mlsanity.com/paper/1911.07508/) (2019)
- [Principled Analyses and Design of First-Order Methods with Inexact Proximal Operators](https://mlsanity.com/paper/2006.06041/) (2020)
- [A sparse Kaczmarz solver and a linearized Bregman method for online compressed sensing](https://mlsanity.com/paper/1403.7543/) (2014)
- [Linear convergence of the Randomized Sparse Kaczmarz Method](https://mlsanity.com/paper/1610.02889/) (2016)
- [Don't relax: early stopping for convex regularization](https://mlsanity.com/paper/1707.05422/) (2017)
- [Dual Extrapolation for Sparse Generalized Linear Models](https://mlsanity.com/paper/1907.05830/) (2019)
- [The Linearized Bregman Method via Split Feasibility Problems: Analysis and Generalizations](https://mlsanity.com/paper/1309.2094/) (2013)
- [Low Complexity Regularization of Linear Inverse Problems](https://mlsanity.com/paper/1407.1598/) (2014)
- [Iterative regularization via dual diagonal descent](https://mlsanity.com/paper/1610.02170/) (2016)
- [Accelerated iterative regularization via dual diagonal descent](https://mlsanity.com/paper/1912.12153/) (2019)

## Built on by (most cited first, 1 of 1)

- [A Nonmonotone Gradient-Based Algorithm for Symmetric Nonnegative Matrix Factorization and Graph Clustering](https://mlsanity.com/paper/2606.02887/) (2026)

## Abstract (arXiv)

Iterative regularization exploits the implicit bias of an optimization algorithm to regularize ill-posed problems. Constructing algorithms with such built-in regularization mechanisms is a classic challenge in inverse problems but also in modern machine learning, where it provides both a new perspective on algorithms analysis, and significant speed-ups compared to explicit regularization. In this work, we propose and study the first iterative regularization procedure able to handle biases described by non smooth and non strongly convex functionals, prominent in low-complexity regularization. Our approach is based on a primal-dual algorithm of which we analyze convergence and stability properties, even in the case where the original problem is unfeasible. The general results are illustrated considering the special case of sparse recovery with the ℓ1 penalty. Our theoretical results are complemented by experiments showing the computational benefits of our approach.