# Compressive Sensing and Neural Networks from a Statistical Learning Perspective

arXiv: https://arxiv.org/abs/2010.15658 · HTML: https://mlsanity.com/paper/2010.15658/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2020.

## Built on by (most cited first, 15 of 15)

- [Theoretical Perspectives on Deep Learning Methods in Inverse Problems](https://mlsanity.com/paper/2206.14373/) (2022)
- [Generalization Error Bounds for Iterative Recovery Algorithms Unfolded as Neural Networks](https://mlsanity.com/paper/2112.04364/) (2021)
- [ADMM-DAD net: a deep unfolding network for analysis compressed sensing](https://mlsanity.com/paper/2110.06986/) (2021)
- [Deep Unfolding: Recent Developments, Theory, and Design Guidelines](https://mlsanity.com/paper/2512.03768/) (2025)
- [Generalization and Estimation Error Bounds for Model-based Neural Networks](https://mlsanity.com/paper/2304.09802/) (2023)
- [DECONET: an Unfolding Network for Analysis-based Compressed Sensing with Generalization Error Bounds](https://mlsanity.com/paper/2205.07050/) (2022)
- [Optimization Guarantees of Unfolded ISTA and ADMM Networks With Smooth Soft-Thresholding](https://mlsanity.com/paper/2309.06195/) (2023)
- [A Generalization Bound for a Family of Implicit Networks](https://mlsanity.com/paper/2410.07427/) (2024)
- [Generalization analysis of an unfolding network for analysis-based Compressed Sensing](https://mlsanity.com/paper/2303.05582/) (2023)
- [Deep greedy unfolding: Sorting out argsorting in greedy sparse recovery algorithms](https://mlsanity.com/paper/2505.15661/) (2025)
- [On Generalization Bounds for Deep Compound Gaussian Neural Networks](https://mlsanity.com/paper/2402.13106/) (2024)
- [Variational Learning ISTA](https://mlsanity.com/paper/2407.06646/) (2024)
- [Unrolled denoising networks provably learn optimal Bayesian inference](https://mlsanity.com/paper/2409.12947/) (2024)
- [ReQuestNet: A Foundational Learning model for Channel Estimation](https://mlsanity.com/paper/2508.08790/) (2025)
- [Adversarial generalization of unfolding (model-based) networks](https://mlsanity.com/paper/2509.15370/) (2025)

## Abstract (arXiv)

Various iterative reconstruction algorithms for inverse problems can be unfolded as neural networks. Empirically, this approach has often led to improved results, but theoretical guarantees are still scarce. While some progress on generalization properties of neural networks have been made, great challenges remain. In this chapter, we discuss and combine these topics to present a generalization error analysis for a class of neural networks suitable for sparse reconstruction from few linear measurements. The hypothesis class considered is inspired by the classical iterative soft-thresholding algorithm (ISTA). The neural networks in this class are obtained by unfolding iterations of ISTA and learning some of the weights. Based on training samples, we aim at learning the optimal network parameters via empirical risk minimization and thereby the optimal network that reconstructs signals from their compressive linear measurements. In particular, we may learn a sparsity basis that is shared by all of the iterations/layers and thereby obtain a new approach for dictionary learning. For this class of networks, we present a generalization bound, which is based on bounding the Rademacher complexity of hypothesis classes consisting of such deep networks via Dudley's integral. Remarkably, under realistic conditions, the generalization error scales only logarithmically in the number of layers, and at most linear in number of measurements.