# Forward Stability of ResNet and Its Variants

arXiv: https://arxiv.org/abs/1811.09885 · HTML: https://mlsanity.com/paper/1811.09885/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2018.

## Built on by (most cited first, 12 of 12)

- [Connections between Numerical Algorithms for PDEs and Neural Networks](https://mlsanity.com/paper/2107.14742/) (2021)
- [Conditioning of Random Feature Matrices: Double Descent and Generalization Error](https://mlsanity.com/paper/2110.11477/) (2021)
- [Translating Numerical Concepts for PDEs into Neural Architectures](https://mlsanity.com/paper/2103.15419/) (2021)
- [Designing Rotationally Invariant Neural Networks from PDEs and Variational Methods](https://mlsanity.com/paper/2108.13993/) (2021)
- [Quantized Convolutional Neural Networks Through the Lens of Partial Differential Equations](https://mlsanity.com/paper/2109.00095/) (2021)
- [Deep Neural Networks Inspired by Differential Equations](https://mlsanity.com/paper/2510.09685/) (2025)
- [Zero Stability Well Predicts Performance of Convolutional Neural Networks](https://mlsanity.com/paper/2206.13100/) (2022)
- [Understand the Effectiveness of Shortcuts through the Lens of DCA](https://mlsanity.com/paper/2412.09853/) (2024)
- [Mathematical Modeling and Convergence Analysis of Deep Neural Networks with Dense Layer Connectivities in Deep Learning](https://mlsanity.com/paper/2510.02049/) (2025)
- [Stability of implicit neural networks for long-term forecasting in dynamical systems](https://mlsanity.com/paper/2305.17155/) (2023)
- [Learning Stochastic Dynamical Systems as an Implicit Regularization with Graph Neural Networks](https://mlsanity.com/paper/2307.06097/) (2023)
- [Deep-layer limit and stability analysis of the basic forward-backward-splitting induced network (II): learning problems](https://mlsanity.com/paper/2605.27133/) (2026)

## Abstract (arXiv)

The residual neural network (ResNet) is a popular deep network architecture which has the ability to obtain high-accuracy results on several image processing problems. In order to analyze the behavior and structure of ResNet, recent work has been on establishing connections between ResNets and continuous-time optimal control problems. In this work, we show that the post-activation ResNet is related to an optimal control problem with differential inclusions, and provide continuous-time stability results for the differential inclusion associated with ResNet. Motivated by the stability conditions, we show that alterations of either the architecture or the optimization problem can generate variants of ResNet which improve the theoretical stability bounds. In addition, we establish stability bounds for the full (discrete) network associated with two variants of ResNet, in particular, bounds on the growth of the features and a measure of the sensitivity of the features with respect to perturbations. These results also help to show the relationship between the depth, regularization, and stability of the feature space. Computational experiments on the proposed variants show that the accuracy of ResNet is preserved and that the accuracy seems to be monotone with respect to the depth and various corruptions.