mlsanityData to 7 Oct 2026

On the Generalization Behavior of Deep Residual Networks From a Dynamical System Perspective

arXiv 2602.20921 ↗First posted 24 Feb 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) have significantly advanced machine learning, with model depth playing a central role in their successes. The dynamical system modeling approach has recently emerged as a powerful framework, offering new mathematical insights into the structure and learning behavior of DNNs. In this work, we establish generalization error bounds for both discrete- and continuous-time residual networks (ResNets) by combining Rademacher complexity, flow maps of dynamical systems, and the convergence behavior of ResNets in the deep-layer limit. The resulting bounds are of order O(1/√S) with respect to the number of training samples S, and include a structure-dependent negative term, yielding depth-uniform and asymptotic generalization bounds under milder assumptions. These findings provide a unified understanding of generalization across both discrete- and continuous-time ResNets, helping to close the gap in both the order of sample complexity and assumptions between the discrete- and continuous-time settings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 45 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

25 of 45 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018Neural Ordinary Differential Equationscited by 3,733
  3. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  4. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  5. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  6. 2018Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Networkcited by 415
  7. 2017Stable Architectures for Deep Neural Networkscited by 340
  8. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
  9. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  10. 2017Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equationscited by 203
  11. 2018Deep Neural Networks Motivated by Partial Differential Equationscited by 187
  12. 2016A vector-contraction inequality for Rademacher complexitiescited by 141
13 more
  1. 2017Data-Dependent Stability of Stochastic Gradient Descentcited by 90
  2. 2019Deep Learning via Dynamical Systems: An Approximation Perspectivecited by 78
  3. 2020Total Deep Variation for Linear Inverse Problemscited by 53
  4. 2016Generalization Error of Invariant Classifierscited by 42
  5. 2019Why ResNet Works? Residuals Generalizecited by 39
  6. 2018Layer-Parallel Training of Deep Residual Neural Networkscited by 35
  7. 2019Deep learning as optimal control problems: models and numerical methodscited by 33
  8. 2018On Tighter Generalization Bound for Deep Neural Networks: CNNs, ResNets, and Beyondcited by 24
  9. 2023Generalization bounds for neural ordinary differential equations and deep residual networkscited by 21
  10. 2021Generalization Error Bounds for Iterative Recovery Algorithms Unfolded as Neural Networkscited by 11
  11. 2023Generalization and Estimation Error Bounds for Model-based Neural Networkscited by 6
  12. 2024Rademacher Complexity of Neural ODEs via Chen-Fliess Seriescited by 5
  13. 2020Overcoming Overfitting and Large Weight Update Problem in Linear Rectifiers: Thresholded Exponential Rectified Linear Unitscited by 1

Built on by

Not cited yet.