# Hyperparameter Tuning is All You Need for LISTA

arXiv: https://arxiv.org/abs/2110.15900 · HTML: https://mlsanity.com/paper/2110.15900/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 29 Oct 2021.

## Builds on (most cited first, 16 of 16)

- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds](https://mlsanity.com/paper/1808.10038/) (2018)
- [FISTA-Net: Learning A Fast Iterative Shrinkage Thresholding Network for Inverse Problems in Imaging](https://mlsanity.com/paper/2008.02683/) (2020)
- [Learned Robust PCA: A Scalable Deep Unfolding Approach for High-Dimensional Outlier Detection](https://mlsanity.com/paper/2110.05649/) (2021)
- [Learning step sizes for unfolded sparse coding](https://mlsanity.com/paper/1905.11071/) (2019)
- [Maximal Sparsity with Deep Networks?](https://mlsanity.com/paper/1605.01636/) (2016)
- [Ada-LISTA: Learned Solvers Adaptive to Varying Models](https://mlsanity.com/paper/2001.08456/) (2020)
- [Tradeoffs between Convergence Speed and Reconstruction Accuracy in Inverse Problems](https://mlsanity.com/paper/1605.09232/) (2016)
- [Understanding Trainable Sparse Coding via Matrix Factorization](https://mlsanity.com/paper/1609.00285/) (2016)
- [LSALSA: Accelerated Source Separation via Learned Sparse Coding](https://mlsanity.com/paper/1802.06875/) (2018)
- [A fast patch-dictionary method for whole image recovery](https://mlsanity.com/paper/1408.3740/) (2014)
- [Neurally Augmented ALISTA](https://mlsanity.com/paper/2010.01930/) (2020)
- [Optimized Projections for Compressed Sensing via Direct Mutual Coherence Minimization](https://mlsanity.com/paper/1508.03117/) (2015)
- [A Design Space Study for LISTA and Beyond](https://mlsanity.com/paper/2104.04110/) (2021)

## Built on by (most cited first, 22 of 22)

- [Theoretical Perspectives on Deep Learning Methods in Inverse Problems](https://mlsanity.com/paper/2206.14373/) (2022)
- [Symbolic Learning to Optimize: Towards Interpretability and Scalability](https://mlsanity.com/paper/2203.06578/) (2022)
- [Towards Constituting Mathematical Structures for Learning to Optimize](https://mlsanity.com/paper/2305.18577/) (2023)
- [Deep Unfolding: Recent Developments, Theory, and Design Guidelines](https://mlsanity.com/paper/2512.03768/) (2025)
- [Nest-DGIL: Nesterov-optimized Deep Geometric Incremental Learning for CS Image Reconstruction](https://mlsanity.com/paper/2308.03807/) (2023)
- [Data-Driven Performance Guarantees for Classical and Learned Optimizers](https://mlsanity.com/paper/2404.13831/) (2024)
- [DISTA-Net: Dynamic Closely-Spaced Infrared Small Target Unmixing](https://mlsanity.com/paper/2505.19148/) (2025)
- [M-L2O: Towards Generalizable Learning-to-Optimize by Test-Time Fast Self-Adaptation](https://mlsanity.com/paper/2303.00039/) (2023)
- [Optimization Guarantees of Unfolded ISTA and ADMM Networks With Smooth Soft-Thresholding](https://mlsanity.com/paper/2309.06195/) (2023)
- [Unveiling and Mitigating Adversarial Vulnerabilities in Iterative Optimizers](https://mlsanity.com/paper/2504.19000/) (2025)
- [Deep FlexQP: Accelerated Nonlinear Programming via Deep Unfolding](https://mlsanity.com/paper/2512.01565/) (2025)
- [Non-Asymptotic Uncertainty Quantification in High-Dimensional Learning](https://mlsanity.com/paper/2407.13666/) (2024)
- [Scalable Deep Unfolding of Conic Optimizers](https://mlsanity.com/paper/2606.13825/) (2026)
- [Variational Learning ISTA](https://mlsanity.com/paper/2407.06646/) (2024)
- [Unrolled denoising networks provably learn optimal Bayesian inference](https://mlsanity.com/paper/2409.12947/) (2024)
- [Physics-Inspired Binary Neural Networks: Interpretable Compression with Theoretical Guarantees](https://mlsanity.com/paper/2502.01908/) (2025)
- [Fast sparse optimization via adaptive shrinkage](https://mlsanity.com/paper/2501.12236/) (2025)
- [Learning Provably Improves the Convergence of Gradient Descent](https://mlsanity.com/paper/2501.18092/) (2025)
- [Tunable Domain Adaptation Using Unfolding](https://mlsanity.com/paper/2603.26931/) (2026)
- [Feature Starvation as Geometric Instability in Sparse Autoencoders](https://mlsanity.com/paper/2605.05341/) (2026)
- [Sparse Koopman Autoencoders Identify Local Dynamical Regimes in Multibasin Systems](https://mlsanity.com/paper/2608.29057/) (2026)
- [DISTA-Net++: Rethinking Infrared Small Target Unmixing Beyond Sub-Pixel Separation](https://mlsanity.com/paper/2609.18773/) (2026)

## Abstract (arXiv)

Learned Iterative Shrinkage-Thresholding Algorithm (LISTA) introduces the concept of unrolling an iterative algorithm and training it like a neural network. It has had great success on sparse recovery. In this paper, we show that adding momentum to intermediate variables in the LISTA network achieves a better convergence rate and, in particular, the network with instance-optimal parameters is superlinearly convergent. Moreover, our new theoretical results lead to a practical approach of automatically and adaptively calculating the parameters of a LISTA network layer based on its previous layers. Perhaps most surprisingly, such an adaptive-parameter procedure reduces the training of LISTA to tuning only three hyperparameters from data: a new record set in the context of the recent advances on trimming down LISTA complexity. We call this new ultra-light weight network HyperLISTA. Compared to state-of-the-art LISTA models, HyperLISTA achieves almost the same performance on seen data distributions and performs better when tested on unseen distributions (specifically, those with different sparsity levels and nonzero magnitudes). Code is available: https://github.com/VITA-Group/HyperLISTA.