mlsanityData to 14 Oct 2026

Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds

arXiv 1808.10038 ↗First posted 29 Aug 2018cs.LG stat.MLcited by 85

Not on the map. Why?

Abstract (arXiv)

In recent years, unfolding iterative algorithms as neural networks has become an empirical success in solving sparse recovery problems. However, its theoretical understanding is still immature, which prevents us from fully utilizing the power of neural networks. In this work, we study unfolded ISTA (Iterative Shrinkage Thresholding Algorithm) for sparse signal recovery. We introduce a weight structure that is necessary for asymptotic convergence to the true sparse signal. With this structure, unfolded ISTA can attain a linear convergence, which is better than the sublinear convergence of ISTA/FISTA in general cases. Furthermore, we propose to incorporate thresholding in the network to perform support selection, which is easy to implement and able to boost the convergence rate both theoretically and empirically. Extensive simulations, including sparse vector recovery and a compressive sensing experiment on real image data, corroborate our theoretical results and demonstrate their practical usefulness. We have made our codes publicly available: https://github.com/xchen-tamu/linear-lista-cpss.

Key facts. Cited by 85 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 21; 2022: 20; 2023: 12; 2024: 15; 2025: 13; 2026: 4.

85 papers.

Sort:

By area:

  1. 2021Learning to Optimize: A Primer and A Benchmarkcited by 208
  2. 2021Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problemcited by 53
  3. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 49
  4. 2022Equivariant Hypergraph Diffusion Neural Operatorscited by 47
  5. 2023Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Modelscited by 39
  6. 2021Learned Robust PCA: A Scalable Deep Unfolding Approach for High-Dimensional Outlier Detectioncited by 34
  7. 2021Hyperparameter Tuning is All You Need for LISTAcited by 22
  8. 2022Theoretical Perspectives on Deep Learning Methods in Inverse Problemscited by 22
  9. 2021Lucas-Kanade Reloaded: End-to-End Super-Resolution from Raw Image Burstscited by 21
  10. 2023Machine Learning for Large-Scale Optimization in 6G Wireless Networkscited by 18
  11. 2021Feasibility-based Fixed Point Networkscited by 17
  12. 2024U-Nets as Belief Propagation: Efficient Classification, Denoising, and Diffusion in Generative Hierarchical Modelscited by 17
38 more
  1. 2022Symbolic Learning to Optimize: Towards Interpretability and Scalabilitycited by 16
  2. 2021Stable and Interpretable Unrolled Dictionary Learningcited by 15
  3. 2022Explainable AI via Learning to Optimizecited by 14
  4. 2023Towards Constituting Mathematical Structures for Learning to Optimizecited by 13
  5. 2021Understanding approximate and unrolled dictionary learning for pattern recoverycited by 11
  6. 2021Generalization Error Bounds for Iterative Recovery Algorithms Unfolded as Neural Networkscited by 11
  7. 2023Differentiating Through Integer Linear Programs with Quadratic Regularization and Davis-Yin Splittingcited by 11
  8. 2023Robust Stochastically-Descending Unrolled Networkscited by 9
  9. 2024Scaling White-Box Transformers for Visioncited by 9
  10. 2022Methods for Recovering Conditional Independence Graphs: A Surveycited by 7
  11. 2025Deep Unfolding: Recent Developments, Theory, and Design Guidelinescited by 7
  12. 2021Deep Algorithm Unrolling for Biomedical Imagingcited by 6
  13. 2023Deep Graph Unfolding for Beamforming in MU-MIMO Interference Networkscited by 6
  14. 2023Stochastic Unrolled Federated Learningcited by 6
  15. 2024Specularity Factorization for Low-Light Enhancementcited by 6
  16. 2024On the Learn-to-Optimize Capabilities of Transformers in In-Context Sparse Recoverycited by 6
  17. 2021Algorithm Unrolling for Massive Access via Deep Neural Network with Theoretical Guaranteecited by 5
  18. 2022Graph-based Algorithm Unfolding for Energy-aware Power Allocation in Wireless Networkscited by 5
  19. 2022Optimization-Derived Learning with Essential Convergence Analysis of Training and Hyper-trainingcited by 5
  20. 2024Data-Driven Performance Guarantees for Classical and Learned Optimizerscited by 5
  21. 2022Hybrid ISTA: Unfolding ISTA With Convergence Guarantees Using Free-Form Deep Neural Networkscited by 4
  22. 2022PAC-Bayesian Learning of Optimization Algorithmscited by 4
  23. 2023Optimization Guarantees of Unfolded ISTA and ADMM Networks With Smooth Soft-Thresholdingcited by 4
  24. 2024ODE-based Learning to Optimizecited by 4
  25. 2025NPN: Non-Linear Projections of the Null-Space for Imaging Inverse Problemscited by 4
  26. 2025Deep FlexQP: Accelerated Nonlinear Programming via Deep Unfoldingcited by 4
  27. 2021Phase Retrieval using Expectation Consistent Signal Recovery Algorithm based on Hypernetworkcited by 3
  28. 2021On the convergence of group-sparse autoencoderscited by 3
  29. 2021Unrolling PALM for sparse semi-blind source separationcited by 3
  30. 2022Bregman Plug-and-Play Priorscited by 3
  31. 2022Learning Variational Models with Unrolling and Bilevel Optimizationcited by 3
  32. 2023Hierarchical Optimization-Derived Learningcited by 3
  33. 2024Solving Inverse Problems with Model Mismatch using Untrained Neural Networks within Model-based Architecturescited by 3
  34. 2025Unsupervised Deformable Image Registration with Structural Nonparametric Smoothingcited by 3
  35. 2025Mechanistic Interpretability with Sparse Autoencoder Neural Operatorscited by 3
  36. 2025Diffusion Generative Models Meet Compressed Sensing, with Applications to Imaging and Financecited by 3
  37. 2021Model-inspired Deep Learning for Light-Field Microscopy with Application to Neuron Localizationcited by 2
  38. 2021CDLNet: Noise-Adaptive Convolutional Dictionary Learning Network for Blind Denoising and Demosaicingcited by 2

50 most cited of 85 All 85 (JSON)