# Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds

arXiv: https://arxiv.org/abs/1808.10038 · HTML: https://mlsanity.com/paper/1808.10038/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Aug 2018.

## Built on by (most cited first, 30 of 85)

- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problem](https://mlsanity.com/paper/2101.08286/) (2021)
- [From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit](https://mlsanity.com/paper/2506.03093/) (2025)
- [Equivariant Hypergraph Diffusion Neural Operators](https://mlsanity.com/paper/2207.06680/) (2022)
- [Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Models](https://mlsanity.com/paper/2309.11420/) (2023)
- [Learned Robust PCA: A Scalable Deep Unfolding Approach for High-Dimensional Outlier Detection](https://mlsanity.com/paper/2110.05649/) (2021)
- [Hyperparameter Tuning is All You Need for LISTA](https://mlsanity.com/paper/2110.15900/) (2021)
- [Theoretical Perspectives on Deep Learning Methods in Inverse Problems](https://mlsanity.com/paper/2206.14373/) (2022)
- [Lucas-Kanade Reloaded: End-to-End Super-Resolution from Raw Image Bursts](https://mlsanity.com/paper/2104.06191/) (2021)
- [Machine Learning for Large-Scale Optimization in 6G Wireless Networks](https://mlsanity.com/paper/2301.03377/) (2023)
- [Feasibility-based Fixed Point Networks](https://mlsanity.com/paper/2104.14090/) (2021)
- [U-Nets as Belief Propagation: Efficient Classification, Denoising, and Diffusion in Generative Hierarchical Models](https://mlsanity.com/paper/2404.18444/) (2024)
- [Symbolic Learning to Optimize: Towards Interpretability and Scalability](https://mlsanity.com/paper/2203.06578/) (2022)
- [Stable and Interpretable Unrolled Dictionary Learning](https://mlsanity.com/paper/2106.00058/) (2021)
- [Explainable AI via Learning to Optimize](https://mlsanity.com/paper/2204.14174/) (2022)
- [Towards Constituting Mathematical Structures for Learning to Optimize](https://mlsanity.com/paper/2305.18577/) (2023)
- [Understanding approximate and unrolled dictionary learning for pattern recovery](https://mlsanity.com/paper/2106.06338/) (2021)
- [Generalization Error Bounds for Iterative Recovery Algorithms Unfolded as Neural Networks](https://mlsanity.com/paper/2112.04364/) (2021)
- [Differentiating Through Integer Linear Programs with Quadratic Regularization and Davis-Yin Splitting](https://mlsanity.com/paper/2301.13395/) (2023)
- [Robust Stochastically-Descending Unrolled Networks](https://mlsanity.com/paper/2312.15788/) (2023)
- [Scaling White-Box Transformers for Vision](https://mlsanity.com/paper/2405.20299/) (2024)
- [Methods for Recovering Conditional Independence Graphs: A Survey](https://mlsanity.com/paper/2211.06829/) (2022)
- [Deep Unfolding: Recent Developments, Theory, and Design Guidelines](https://mlsanity.com/paper/2512.03768/) (2025)
- [Deep Algorithm Unrolling for Biomedical Imaging](https://mlsanity.com/paper/2108.06637/) (2021)
- [Deep Graph Unfolding for Beamforming in MU-MIMO Interference Networks](https://mlsanity.com/paper/2304.00446/) (2023)
- [Stochastic Unrolled Federated Learning](https://mlsanity.com/paper/2305.15371/) (2023)
- [Specularity Factorization for Low-Light Enhancement](https://mlsanity.com/paper/2404.01998/) (2024)
- [On the Learn-to-Optimize Capabilities of Transformers in In-Context Sparse Recovery](https://mlsanity.com/paper/2410.13981/) (2024)
- [Algorithm Unrolling for Massive Access via Deep Neural Network with Theoretical Guarantee](https://mlsanity.com/paper/2106.10426/) (2021)
- [Graph-based Algorithm Unfolding for Energy-aware Power Allocation in Wireless Networks](https://mlsanity.com/paper/2201.11799/) (2022)

## Abstract (arXiv)

In recent years, unfolding iterative algorithms as neural networks has become an empirical success in solving sparse recovery problems. However, its theoretical understanding is still immature, which prevents us from fully utilizing the power of neural networks. In this work, we study unfolded ISTA (Iterative Shrinkage Thresholding Algorithm) for sparse signal recovery. We introduce a weight structure that is necessary for asymptotic convergence to the true sparse signal. With this structure, unfolded ISTA can attain a linear convergence, which is better than the sublinear convergence of ISTA/FISTA in general cases. Furthermore, we propose to incorporate thresholding in the network to perform support selection, which is easy to implement and able to boost the convergence rate both theoretically and empirically. Extensive simulations, including sparse vector recovery and a compressive sensing experiment on real image data, corroborate our theoretical results and demonstrate their practical usefulness. We have made our codes publicly available: https://github.com/xchen-tamu/linear-lista-cpss.