# Fast sparse optimization via adaptive shrinkage

arXiv: https://arxiv.org/abs/2501.12236 · HTML: https://mlsanity.com/paper/2501.12236/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 28 references that are in the map.
- First posted on arXiv 21 Jan 2025.

## Builds on (most cited first, 4 of 4)

- [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017)
- [Hyperparameter Tuning is All You Need for LISTA](https://mlsanity.com/paper/2110.15900/) (2021)
- [On the Convergence of the Iterative Shrinkage/Thresholding Algorithm With a Weakly Convex Penalty](https://mlsanity.com/paper/1510.07821/) (2015)
- [Centralized and distributed online learning for sparse time-varying optimization](https://mlsanity.com/paper/2001.11939/) (2020)

## Abstract (arXiv)

The need for fast sparse optimization is emerging, e.g., to deal with large-dimensional data-driven problems and to track time-varying systems. In the framework of linear sparse optimization, the iterative shrinkage-thresholding algorithm is a valuable method to solve Lasso, which is particularly appreciated for its ease of implementation. Nevertheless, it converges slowly. In this paper, we develop a proximal method, based on logarithmic regularization, which turns out to be an iterative shrinkage-thresholding algorithm with adaptive shrinkage hyperparameter. This adaptivity substantially enhances the trajectory of the algorithm, in a way that yields faster convergence, while keeping the simplicity of the original method. Our contribution is twofold: on the one hand, we derive and analyze the proposed algorithm; on the other hand, we validate its fast convergence via numerical experiments and we discuss the performance with respect to state-of-the-art algorithms.