# Learning step sizes for unfolded sparse coding

arXiv: https://arxiv.org/abs/1905.11071 · HTML: https://mlsanity.com/paper/1905.11071/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2019.

## Built on by (most cited first, 30 of 30)

- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit](https://mlsanity.com/paper/2506.03093/) (2025)
- [Learning to Learn Graph Topologies](https://mlsanity.com/paper/2110.09807/) (2021)
- [Hyperparameter Tuning is All You Need for LISTA](https://mlsanity.com/paper/2110.15900/) (2021)
- [Theoretical Perspectives on Deep Learning Methods in Inverse Problems](https://mlsanity.com/paper/2206.14373/) (2022)
- [Machine Learning for Large-Scale Optimization in 6G Wireless Networks](https://mlsanity.com/paper/2301.03377/) (2023)
- [Stable and Interpretable Unrolled Dictionary Learning](https://mlsanity.com/paper/2106.00058/) (2021)
- [Towards Constituting Mathematical Structures for Learning to Optimize](https://mlsanity.com/paper/2305.18577/) (2023)
- [Understanding approximate and unrolled dictionary learning for pattern recovery](https://mlsanity.com/paper/2106.06338/) (2021)
- [Algorithm Unrolling for Massive Access via Deep Neural Network with Theoretical Guarantee](https://mlsanity.com/paper/2106.10426/) (2021)
- [Optimization-Derived Learning with Essential Convergence Analysis of Training and Hyper-training](https://mlsanity.com/paper/2206.07875/) (2022)
- [Unrolling PALM for sparse semi-blind source separation](https://mlsanity.com/paper/2112.05694/) (2021)
- [Learning Variational Models with Unrolling and Bilevel Optimization](https://mlsanity.com/paper/2209.12651/) (2022)
- [Hierarchical Optimization-Derived Learning](https://mlsanity.com/paper/2302.05587/) (2023)
- [Mechanistic Interpretability with Sparse Autoencoder Neural Operators](https://mlsanity.com/paper/2509.03738/) (2025)
- [Learned ISTA with Error-based Thresholding for Adaptive Sparse Coding](https://mlsanity.com/paper/2112.10985/) (2021)
- [Uncertainty quantification for learned ISTA](https://mlsanity.com/paper/2309.07982/) (2023)
- [Learning Interpretable Deep Disentangled Neural Networks for Hyperspectral Unmixing](https://mlsanity.com/paper/2310.02340/) (2023)
- [From Learning to Optimize to Learning Optimization Algorithms](https://mlsanity.com/paper/2405.18222/) (2024)
- [Accelerating Multi-Block Constrained Optimization Through Learning to Optimize](https://mlsanity.com/paper/2409.17320/) (2024)
- [A Design Space Study for LISTA and Beyond](https://mlsanity.com/paper/2104.04110/) (2021)
- [Gabor is Enough: Interpretable Deep Denoising with a Gabor Synthesis Dictionary Prior](https://mlsanity.com/paper/2204.11146/) (2022)
- [Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Models](https://mlsanity.com/paper/2306.03249/) (2023)
- [Evaluating Sparse Autoencoders: From Shallow Design to Matching Pursuit](https://mlsanity.com/paper/2506.05239/) (2025)
- [Deep Learning Techniques for Compressive Sensing-Based Reconstruction and Inference -- A Ubiquitous Systems Perspective](https://mlsanity.com/paper/2105.13191/) (2021)
- [Learned Interpretable Residual Extragradient ISTA for Sparse Coding](https://mlsanity.com/paper/2106.11970/) (2021)
- [Active Inference in Hebbian Learning Networks](https://mlsanity.com/paper/2306.05053/) (2023)
- [Analysis and Synthesis Denoisers for Forward-Backward Plug-and-Play Algorithms](https://mlsanity.com/paper/2411.13276/) (2024)
- [Accelerating Hierarchical Sparse Predictive Coding with Hybrid Amortized Inference](https://mlsanity.com/paper/2606.27802/) (2026)
- [Deeply unrolled alternating projections for phase retrieval](https://mlsanity.com/paper/2610.17316/) (2026)

## Abstract (arXiv)

Sparse coding is typically solved by iterative optimization techniques, such as the Iterative Shrinkage-Thresholding Algorithm (ISTA). Unfolding and learning weights of ISTA using neural networks is a practical way to accelerate estimation. In this paper, we study the selection of adapted step sizes for ISTA. We show that a simple step size strategy can improve the convergence rate of ISTA by leveraging the sparsity of the iterates. However, it is impractical in most large-scale applications. Therefore, we propose a network architecture where only the step sizes of ISTA are learned. We demonstrate that for a large class of unfolded algorithms, if the algorithm converges to the solution of the Lasso, its last layers correspond to ISTA with learned step sizes. Experiments show that our method is competitive with state-of-the-art networks when the solutions are sparse enough.