mlsanityData to 7 Oct 2026

Generalization bounds for neural ordinary differential equations and deep residual networks

arXiv 2305.06648 ↗First posted 11 May 2023stat.ML cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

Neural ordinary differential equations (neural ODEs) are a popular family of continuous-depth deep learning models. In this work, we consider a large family of parameterized ODEs with continuous-in-time parameters, which include time-dependent neural ODEs. We derive a generalization bound for this class by a Lipschitz-based argument. By leveraging the analogy between neural ODEs and deep residual networks, our approach yields in particular a generalization bound for a class of deep residual networks. The bound involves the magnitude of the difference between successive weight matrices. We illustrate numerically how this quantity affects the generalization capability of neural networks.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 32 of its 52 references that are in the map. First posted on arXiv 11 May 2023.

Builds on

32 of 52 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  6. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  7. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  8. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  9. 2017Stable Architectures for Deep Neural Networkscited by 340
  10. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
  11. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 304
  12. 2015Norm-Based Capacity Control in Neural Networkscited by 299
20 more
  1. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  2. 2017Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equationscited by 203
  3. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matchingcited by 182
  4. 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 168
  5. 2020Dissecting Neural ODEscited by 150
  6. 2021Neural Flows: Efficient Alternative to Neural ODEscited by 99
  7. 2020Learning Differential Equations that are Easy to Solvecited by 81
  8. 2021Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease Progressioncited by 48
  9. 2021Noisy Recurrent Neural Networkscited by 42
  10. 2020Continuous-in-Depth Neural Networkscited by 40
  11. 2021LEADS: Learning Dynamical Systems that Generalize Across Environmentscited by 39
  12. 2021Pay attention to your loss: understanding misconceptions about 1-Lipschitz neural networkscited by 30
  13. 2022Do Residual Neural Networks discretize Neural Ordinary Differential Equations?cited by 27
  14. 2021Framing RNN as a kernel method: A neural ODE approachcited by 23
  15. 2020Robust pricing and hedging via neural SDEscited by 21
  16. 2022Scaling ResNets in the Large-depth Regimecited by 18
  17. 2021Scaling Properties of Deep Residual Networkscited by 16
  18. 2021Stateful ODE-Nets using Basis Function Expansionscited by 14
  19. 2022Fitting an immersed submanifold to data via Sussmann's orbit theoremcited by 4
  20. 2022Neural Generalized Ordinary Differential Equations with Layer-varying Parameterscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 7; 2025: 5; 2026: 5.

21 papers.

Sort:

  1. 2024Large Language Models as Markov Chainscited by 33
  2. 2023Implicit regularization of deep residual networks towards neural ODEscited by 20
  3. 2023Gaussian Interpolation Flowscited by 17
  4. 2024Topological Neural Networks go Persistent, Equivariant, and Continuouscited by 15
  5. 2024Feedback Favors the Generalization of Neural ODEscited by 8
  6. 2026On the Residual Scaling of Looped Transformers: Stability and Transferabilitycited by 7
  7. 2024Rademacher Complexity of Neural ODEs via Chen-Fliess Seriescited by 5
  8. 2024A Generalization Bound for a Family of Implicit Networkscited by 4
  9. 2023On the Generalization and Approximation Capacities of Neural Controlled Differential Equationscited by 3
  10. 2025Bridging Neural ODE and ResNet: A Formal Error Bound for Safety Verificationcited by 3
  11. 2024Length independent generalization bounds for deep SSM architectures via Rademacher contraction and stability constraintscited by 2
  12. 2025Statistical physics analysis of graph neural networks: Approaching optimality in the contextual stochastic block modelcited by 2
9 more
  1. 2025Generalization Bound for a General Class of Neural Ordinary Differential Equationscited by 1
  2. 2026Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformerscited by 1
  3. 2023PAC bounds of continuous Linear Parameter-Varying systems related to neural ODEscited by 0
  4. 2024A finite-sample generalization bound for stable LPV systemscited by 0
  5. 2025Sharper Risk Bound for Multi-Task Learning with Multi-Graph Dependent Datacited by 0
  6. 2025Dynamical Implicit Neural Representationscited by 0
  7. 2026On the Generalization Behavior of Deep Residual Networks From a Dynamical System Perspectivecited by 0
  8. 2026Upper Generalization Bounds for Neural Oscillatorscited by 0
  9. 2026Constructive interpolation and generalization rates for neural ODEs: a control perspectivecited by 0