# Robust Stochastically-Descending Unrolled Networks

arXiv: https://arxiv.org/abs/2312.15788 · HTML: https://mlsanity.com/paper/2312.15788/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 55 references that are in the map.
- First posted on arXiv 25 Dec 2023.

## Builds on (most cited first, 30 of 36)

- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Transformers learn in-context by gradient descent](https://mlsanity.com/paper/2212.07677/) (2022)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [Deep Unfolding Network for Image Super-Resolution](https://mlsanity.com/paper/2003.10428/) (2020)
- [Stability Properties of Graph Neural Networks](https://mlsanity.com/paper/1905.04497/) (2019)
- [Deep Generalized Unfolding Networks for Image Restoration](https://mlsanity.com/paper/2204.13348/) (2022)
- [Invertible generative models for inverse problems: mitigating representation error and dataset bias](https://mlsanity.com/paper/1905.11672/) (2019)
- [Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds](https://mlsanity.com/paper/1808.10038/) (2018)
- [White-Box Transformers via Sparse Rate Reduction](https://mlsanity.com/paper/2306.01129/) (2023)
- [Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures](https://mlsanity.com/paper/1409.2574/) (2014)
- [Unfolding WMMSE using Graph Neural Networks for Efficient Power Allocation](https://mlsanity.com/paper/2009.10812/) (2020)
- [Iterative Algorithm Induced Deep-Unfolding Neural Networks: Precoding Design for Multiuser MIMO Systems](https://mlsanity.com/paper/2006.08099/) (2020)
- [Constrained Learning with Non-Convex Losses](https://mlsanity.com/paper/2103.05134/) (2021)
- [Solving Inverse Problems with a Flow-based Noise Model](https://mlsanity.com/paper/2003.08089/) (2020)
- [Transformers from an Optimization Perspective](https://mlsanity.com/paper/2205.13891/) (2022)
- [Safeguarded Learned Convex Optimization](https://mlsanity.com/paper/2003.01880/) (2020)
- [Deep Unfolding with Normalizing Flow Priors for Inverse Problems](https://mlsanity.com/paper/2107.02848/) (2021)
- [InDuDoNet+: A Deep Unfolding Dual Domain Network for Metal Artifact Reduction in CT Images](https://mlsanity.com/paper/2112.12660/) (2021)
- [Differentiable Linearized ADMM](https://mlsanity.com/paper/1905.06179/) (2019)
- [Controlling Neural Networks via Energy Dissipation](https://mlsanity.com/paper/1904.03081/) (2019)
- [Space-Time Graph Neural Networks](https://mlsanity.com/paper/2110.02880/) (2021)
- [Coordinated Sum-Rate Maximization in Multicell MU-MIMO with Deep Unrolling](https://mlsanity.com/paper/2202.10371/) (2022)
- [Deep Algorithm Unrolling for Biomedical Imaging](https://mlsanity.com/paper/2108.06637/) (2021)
- [Stochastic Unrolled Federated Learning](https://mlsanity.com/paper/2305.15371/) (2023)
- [Optimization-Derived Learning with Essential Convergence Analysis of Training and Hyper-training](https://mlsanity.com/paper/2206.07875/) (2022)
- [Space-Time Graph Neural Networks with Stochastic Graph Perturbations](https://mlsanity.com/paper/2210.16270/) (2022)

## Built on by (most cited first, 9 of 9)

- [Proximal Algorithm Unrolling: Flexible and Efficient Reconstruction Networks for Single-Pixel Imaging](https://mlsanity.com/paper/2505.23180/) (2025)
- [Deep Unfolding: Recent Developments, Theory, and Design Guidelines](https://mlsanity.com/paper/2512.03768/) (2025)
- [GKNet: Graph Kalman Filtering and Model Inference via Model-based Deep Learning](https://mlsanity.com/paper/2506.22004/) (2025)
- [Unrolled Graph Neural Networks for Constrained Optimization](https://mlsanity.com/paper/2509.17156/) (2025)
- [Unrolling Dynamic Programming via Graph Filters](https://mlsanity.com/paper/2507.21705/) (2025)
- [A Constrained Optimization Perspective of Unrolled Transformers](https://mlsanity.com/paper/2601.17257/) (2026)
- [Unrolled Neural Networks for Constrained Optimization](https://mlsanity.com/paper/2601.17274/) (2026)
- [Sparse Bayesian Learning Algorithms Revisited: From Learning Majorizers to Structured Algorithmic Learning using Neural Networks](https://mlsanity.com/paper/2604.02513/) (2026)
- [ForeSplat: Optimization-Aware Foresight for Feed-Forward 3D Gaussian Splatting](https://mlsanity.com/paper/2605.22020/) (2026)

## Abstract (arXiv)

Deep unrolling, or unfolding, is an emerging learning-to-optimize method that unrolls a truncated iterative algorithm in the layers of a trainable neural network. However, the convergence guarantees and generalizability of the unrolled networks are still open theoretical problems. To tackle these problems, we provide deep unrolled architectures with a stochastic descent nature by imposing descending constraints during training. The descending constraints are forced layer by layer to ensure that each unrolled layer takes, on average, a descent step toward the optimum during training. We theoretically prove that the sequence constructed by the outputs of the unrolled layers is then guaranteed to converge for unseen problems, assuming no distribution shift between training and test problems. We also show that standard unrolling is brittle to perturbations, and our imposed constraints provide the unrolled networks with robustness to additive noise and perturbations. We numerically assess unrolled architectures trained under the proposed constraints in two different applications, including the sparse coding using learnable iterative shrinkage and thresholding algorithm (LISTA) and image inpainting using proximal generative flow (GLOW-Prox), and demonstrate the performance and robustness benefits of the proposed method.