# NESTANets: Stable, accurate and efficient neural networks for analysis-sparse inverse problems

arXiv: https://arxiv.org/abs/2203.00804 · HTML: https://mlsanity.com/paper/2203.00804/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 97 references that are in the map.
- First posted on arXiv 2 Mar 2022.

## Builds on (most cited first, 27 of 27)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Learned Primal-dual Reconstruction](https://mlsanity.com/paper/1707.06474/) (2017)
- [On instabilities of deep learning in image reconstruction - Does AI come at a cost?](https://mlsanity.com/paper/1902.05300/) (2019)
- [An overview of deep learning in medical imaging focusing on MRI](https://mlsanity.com/paper/1811.10052/) (2018)
- [Deep Equilibrium Architectures for Inverse Problems in Imaging](https://mlsanity.com/paper/2102.07944/) (2021)
- [A Review of Convolutional Neural Networks for Inverse Problems in Imaging](https://mlsanity.com/paper/1710.04011/) (2017)
- [Deep Learning Methods for Parallel Magnetic Resonance Image Reconstruction](https://mlsanity.com/paper/1904.01112/) (2019)
- [On hallucinations in tomographic image reconstruction](https://mlsanity.com/paper/2012.00646/) (2020)
- [Learned D-AMP: Principled Neural Network based Compressive Image Recovery](https://mlsanity.com/paper/1704.06625/) (2017)
- [Image Reconstruction: From Sparsity to Data-adaptive Methods and Machine Learning](https://mlsanity.com/paper/1904.02816/) (2019)
- [Solving Inverse Problems With Deep Neural Networks -- Robustness Included?](https://mlsanity.com/paper/2011.04268/) (2020)
- [Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operators](https://mlsanity.com/paper/1903.01014/) (2019)
- [Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problem](https://mlsanity.com/paper/2101.08286/) (2021)
- [Measuring Robustness in Deep Learning Based Compressive Sensing](https://mlsanity.com/paper/2102.06103/) (2021)
- [Deep J-Sense: Accelerated MRI Reconstruction via Unrolled Alternating Optimization](https://mlsanity.com/paper/2103.02087/) (2021)
- [A Review on Deep Learning in Medical Image Reconstruction](https://mlsanity.com/paper/1906.10643/) (2019)
- [A Simple Nearly-Optimal Restart Scheme For Speeding-Up First Order Methods](https://mlsanity.com/paper/1803.00151/) (2018)
- [Do CNNs solve the CT inverse problem?](https://mlsanity.com/paper/2005.10755/) (2020)
- [A review and experimental evaluation of deep learning methods for MRI reconstruction](https://mlsanity.com/paper/2109.08618/) (2021)
- [Deep BCD-Net Using Identical Encoding-Decoding CNN Structures for Iterative Image Recovery](https://mlsanity.com/paper/1802.07129/) (2018)
- [Limitations of Deep Learning for Inverse Problems on Digital Hardware](https://mlsanity.com/paper/2202.13490/) (2022)
- [Validation and Generalizability of Self-Supervised Image Reconstruction Methods for Undersampled MRI](https://mlsanity.com/paper/2201.12535/) (2022)
- [Equilibrated Zeroth-Order Unrolled Deep Networks for Accelerated MRI](https://mlsanity.com/paper/2112.09891/) (2021)
- [Computational Complexity versus Statistical Performance on Sparse Recovery Problems](https://mlsanity.com/paper/1506.03295/) (2015)

## Built on by (most cited first, 5 of 5)

- [Learning smooth functions in high dimensions: from sparse polynomials to deep neural networks](https://mlsanity.com/paper/2404.03761/) (2024)
- [Implicit regularization in AI meets generalized hardness of approximation in optimization -- Sharp results for diagonal linear networks](https://mlsanity.com/paper/2307.07410/) (2023)
- [Restarts subject to approximate sharpness: A parameter-free and optimal scheme for first-order methods](https://mlsanity.com/paper/2301.02268/) (2023)
- [Do stable neural networks exist for classification problems? -- A new view on stability in AI](https://mlsanity.com/paper/2401.07874/) (2024)
- [How many measurements are enough? Bayesian recovery in inverse problems with general distributions](https://mlsanity.com/paper/2505.10630/) (2025)

## Abstract (arXiv)

Solving inverse problems is a fundamental component of science, engineering and mathematics. With the advent of deep learning, deep neural networks have significant potential to outperform existing state-of-the-art, model-based methods for solving inverse problems. However, it is known that current data-driven approaches face several key issues, notably hallucinations, instabilities and unpredictable generalization, with potential impact in critical tasks such as medical imaging. This raises the key question of whether or not one can construct deep neural networks for inverse problems with explicit stability and accuracy guarantees. In this work, we present a novel construction of accurate, stable and efficient neural networks for inverse problems with general analysis-sparse models, termed NESTANets. To construct the network, we first unroll NESTA, an accelerated first-order method for convex optimization. The slow convergence of this method leads to deep networks with low efficiency. Therefore, to obtain shallow, and consequently more efficient, networks we combine NESTA with a novel restart scheme. We then use compressed sensing techniques to demonstrate accuracy and stability. We showcase this approach in the case of Fourier imaging, and verify its stability and performance via a series of numerical experiments. The key impact of this work is demonstrating the construction of efficient neural networks based on unrolling with guaranteed stability and accuracy.