mlsanityData to 7 Oct 2026

Learning step sizes for unfolded sparse coding

arXiv 1905.11071 ↗First posted 27 May 2019stat.ML cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

Sparse coding is typically solved by iterative optimization techniques, such as the Iterative Shrinkage-Thresholding Algorithm (ISTA). Unfolding and learning weights of ISTA using neural networks is a practical way to accelerate estimation. In this paper, we study the selection of adapted step sizes for ISTA. We show that a simple step size strategy can improve the convergence rate of ISTA by leveraging the sparsity of the iterates. However, it is impractical in most large-scale applications. Therefore, we propose a network architecture where only the step sizes of ISTA are learned. We demonstrate that for a large class of unfolded algorithms, if the algorithm converges to the solution of the Lasso, its last layers correspond to ISTA with learned step sizes. Experiments show that our method is competitive with state-of-the-art networks when the solutions are sparse enough.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 4; 2023: 7; 2024: 3; 2025: 3; 2026: 1.

29 papers.

Sort:

  1. 2021Learning to Optimize: A Primer and A Benchmarkcited by 206
  2. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 45
  3. 2021Learning to Learn Graph Topologiescited by 26
  4. 2021Hyperparameter Tuning is All You Need for LISTAcited by 22
  5. 2022Theoretical Perspectives on Deep Learning Methods in Inverse Problemscited by 22
  6. 2023Machine Learning for Large-Scale Optimization in 6G Wireless Networkscited by 18
  7. 2021Stable and Interpretable Unrolled Dictionary Learningcited by 15
  8. 2023Towards Constituting Mathematical Structures for Learning to Optimizecited by 13
  9. 2021Understanding approximate and unrolled dictionary learning for pattern recoverycited by 11
  10. 2021Algorithm Unrolling for Massive Access via Deep Neural Network with Theoretical Guaranteecited by 5
  11. 2022Optimization-Derived Learning with Essential Convergence Analysis of Training and Hyper-trainingcited by 5
  12. 2021Unrolling PALM for sparse semi-blind source separationcited by 3
17 more
  1. 2022Learning Variational Models with Unrolling and Bilevel Optimizationcited by 3
  2. 2023Hierarchical Optimization-Derived Learningcited by 3
  3. 2025Mechanistic Interpretability with Sparse Autoencoder Neural Operatorscited by 3
  4. 2021Learned ISTA with Error-based Thresholding for Adaptive Sparse Codingcited by 2
  5. 2023Uncertainty quantification for learned ISTAcited by 2
  6. 2023Learning Interpretable Deep Disentangled Neural Networks for Hyperspectral Unmixingcited by 2
  7. 2024From Learning to Optimize to Learning Optimization Algorithmscited by 2
  8. 2024Accelerating Multi-Block Constrained Optimization Through Learning to Optimizecited by 2
  9. 2021A Design Space Study for LISTA and Beyondcited by 1
  10. 2022Gabor is Enough: Interpretable Deep Denoising with a Gabor Synthesis Dictionary Priorcited by 1
  11. 2023Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Modelscited by 1
  12. 2025Evaluating Sparse Autoencoders: From Shallow Design to Matching Pursuitcited by 1
  13. 2021Deep Learning Techniques for Compressive Sensing-Based Reconstruction and Inference -- A Ubiquitous Systems Perspectivecited by 0
  14. 2021Learned Interpretable Residual Extragradient ISTA for Sparse Codingcited by 0
  15. 2023Active Inference in Hebbian Learning Networkscited by 0
  16. 2024Analysis and Synthesis Denoisers for Forward-Backward Plug-and-Play Algorithmscited by 0
  17. 2026Accelerating Hierarchical Sparse Predictive Coding with Hybrid Amortized Inferencecited by 0