mlsanityData to 14 Oct 2026

Neural ODEs as the Deep Limit of ResNets with constant weights

arXiv 1906.12183 ↗First posted 28 Jun 2019stat.ML cs.LG math.APcited by 10

Not on the map. Why?

Abstract (arXiv)

In this paper we prove that, in the deep limit, the stochastic gradient descent on a ResNet type deep neural network, where each layer shares the same weight matrix, converges to the stochastic gradient descent for a Neural ODE and that the corresponding value/loss functions converge. Our result gives, in the context of minimization by stochastic gradient descent, a theoretical foundation for considering Neural ODEs as the deep limit of ResNets. Our proof is based on certain decay estimates for associated Fokker-Planck equations.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). First posted on arXiv 28 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 2; 2023: 1; 2024: 2; 2025: 3.

10 papers.

  1. 2025The Hidden Width of Deep ResNets: Tight Error Bounds and Phase Diagramcited by 17
  2. 2024Unification of Symmetries Inside Neural Networks: Transformer, Feedforward and Neural ODEcited by 7
  3. 2022Learning Subgrid-scale Models with Neural Ordinary Differential Equationscited by 4
  4. 2022Learning via nonlinear conjugate gradients and depth-varying neural ODEscited by 3
  5. 2021Approximation properties of Residual Neural Networks for Kolmogorov PDEscited by 2
  6. 2024The Disappearance of Timestep Embedding in Modern Time-Dependent Neural Networkscited by 2
  7. 2025On the Convergence and Size Transferability of Continuous-depth Graph Neural Networkscited by 1
  8. 2021Deep limits and cut-off phenomena for neural networkscited by 0
  9. 2023Dimensionality Collapse: Optimal Measurement Selection for Low-Error Infinite-Horizon Forecastingcited by 0
  10. 2025A multilevel approach to accelerate the training of Transformerscited by 0