# Generalization Error Bounds for Iterative Recovery Algorithms Unfolded as Neural Networks

arXiv: https://arxiv.org/abs/2112.04364 · HTML: https://mlsanity.com/paper/2112.04364/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 8 Dec 2021.

## Builds on (most cited first, 20 of 20)

- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [Exploring Generalization in Deep Learning](https://mlsanity.com/paper/1706.08947/) (2017)
- [Fantastic Generalization Measures and Where to Find Them](https://mlsanity.com/paper/1912.02178/) (2019)
- [A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks](https://mlsanity.com/paper/1707.09564/) (2017)
- [Stronger generalization bounds for deep nets via a compression approach](https://mlsanity.com/paper/1802.05296/) (2018)
- [Size-Independent Sample Complexity of Neural Networks](https://mlsanity.com/paper/1712.06541/) (2017)
- [A vector-contraction inequality for Rademacher complexities](https://mlsanity.com/paper/1605.00251/) (2016)
- [Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group](https://mlsanity.com/paper/1901.08428/) (2019)
- [Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds](https://mlsanity.com/paper/1808.10038/) (2018)
- [A Deep Learning Approach to Structured Signal Recovery](https://mlsanity.com/paper/1508.04065/) (2015)
- [Convolutional Proximal Neural Networks and Plug-and-Play Algorithms](https://mlsanity.com/paper/2011.02281/) (2020)
- [Maximal Sparsity with Deep Networks?](https://mlsanity.com/paper/1605.01636/) (2016)
- [Learned Convolutional Sparse Coding](https://mlsanity.com/paper/1711.00328/) (2017)
- [Ada-LISTA: Learned Solvers Adaptive to Varying Models](https://mlsanity.com/paper/2001.08456/) (2020)
- [Compressive Sensing and Neural Networks from a Statistical Learning Perspective](https://mlsanity.com/paper/2010.15658/) (2020)
- [Sample Complexity of Dictionary Learning and other Matrix Factorizations](https://mlsanity.com/paper/1312.3790/) (2013)
- [ADMM-DAD net: a deep unfolding network for analysis compressed sensing](https://mlsanity.com/paper/2110.06986/) (2021)
- [On the Minimax Risk of Dictionary Learning](https://mlsanity.com/paper/1507.05498/) (2015)
- [Learning optimal nonlinearities for iterative thresholding algorithms](https://mlsanity.com/paper/1512.04754/) (2015)
- [Performance Limits of Dictionary Learning for Sparse Coding](https://mlsanity.com/paper/1402.4078/) (2014)

## Built on by (most cited first, 11 of 11)

- [Theoretical Perspectives on Deep Learning Methods in Inverse Problems](https://mlsanity.com/paper/2206.14373/) (2022)
- [Deep Unfolding: Recent Developments, Theory, and Design Guidelines](https://mlsanity.com/paper/2512.03768/) (2025)
- [Generalization and Estimation Error Bounds for Model-based Neural Networks](https://mlsanity.com/paper/2304.09802/) (2023)
- [DECONET: an Unfolding Network for Analysis-based Compressed Sensing with Generalization Error Bounds](https://mlsanity.com/paper/2205.07050/) (2022)
- [Optimization Guarantees of Unfolded ISTA and ADMM Networks With Smooth Soft-Thresholding](https://mlsanity.com/paper/2309.06195/) (2023)
- [A Rescaling-Invariant Lipschitz Bound Based on Path-Metrics for Modern ReLU Network Parameterizations](https://mlsanity.com/paper/2405.15006/) (2024)
- [From Score Learning to Discretized Sampling: An End-to-End Generalization Analysis of Diffusion Models](https://mlsanity.com/paper/2607.23226/) (2026)
- [Variational Learning ISTA](https://mlsanity.com/paper/2407.06646/) (2024)
- [Unrolled denoising networks provably learn optimal Bayesian inference](https://mlsanity.com/paper/2409.12947/) (2024)
- [ReQuestNet: A Foundational Learning model for Channel Estimation](https://mlsanity.com/paper/2508.08790/) (2025)
- [On the Generalization Behavior of Deep Residual Networks From a Dynamical System Perspective](https://mlsanity.com/paper/2602.20921/) (2026)

## Abstract (arXiv)

Motivated by the learned iterative soft thresholding algorithm (LISTA), we introduce a general class of neural networks suitable for sparse reconstruction from few linear measurements. By allowing a wide range of degrees of weight-sharing between the layers, we enable a unified analysis for very different neural network types, ranging from recurrent ones to networks more similar to standard feedforward neural networks. Based on training samples, via empirical risk minimization we aim at learning the optimal network parameters and thereby the optimal network that reconstructs signals from their low-dimensional linear measurements. We derive generalization bounds by analyzing the Rademacher complexity of hypothesis classes consisting of such deep networks, that also take into account the thresholding parameters. We obtain estimates of the sample complexity that essentially depend only linearly on the number of parameters and on the depth. We apply our main result to obtain specific generalization bounds for several practical examples, including different algorithms for (implicit) dictionary learning, and convolutional neural networks.