# Large-time asymptotics in deep learning

arXiv: https://arxiv.org/abs/2008.02491 · HTML: https://mlsanity.com/paper/2008.02491/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2020.

## Built on by (most cited first, 14 of 14)

- [Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNs](https://mlsanity.com/paper/2406.02997/) (2024)
- [Interplay between depth and width for interpolation in neural ODEs](https://mlsanity.com/paper/2401.09902/) (2024)
- [Sparsity in long-time control of neural ODEs](https://mlsanity.com/paper/2102.13566/) (2021)
- [Learning via nonlinear conjugate gradients and depth-varying neural ODEs](https://mlsanity.com/paper/2202.05766/) (2022)
- [Variational formulations of ODE-Net as a mean-field optimal control problem and existence results](https://mlsanity.com/paper/2303.05924/) (2023)
- [From NeurODEs to AutoencODEs: a mean-field control framework for width-varying Neural Networks](https://mlsanity.com/paper/2307.02279/) (2023)
- [Normalizing flows as approximations of optimal transport maps via linear-control neural ODEs](https://mlsanity.com/paper/2311.01404/) (2023)
- [The Sherpa.ai Blind Vertical Federated Learning Paradigm to Minimize the Number of Communications](https://mlsanity.com/paper/2510.17901/) (2025)
- [On the Turnpike to Design of Deep Neural Nets: Explicit Depth Bounds](https://mlsanity.com/paper/2101.03000/) (2021)
- [On Dissipativity of Cross-Entropy Loss in Training ResNets](https://mlsanity.com/paper/2405.19013/) (2024)
- [Towards an Optimal Control Perspective of ResNet Training](https://mlsanity.com/paper/2506.21453/) (2025)
- [Mathematical Modeling and Convergence Analysis of Deep Neural Networks with Dense Layer Connectivities in Deep Learning](https://mlsanity.com/paper/2510.02049/) (2025)
- [Convergence, design and training of continuous-time dropout as a random batch method](https://mlsanity.com/paper/2510.13134/) (2025)
- [Constructive interpolation and generalization rates for neural ODEs: a control perspective](https://mlsanity.com/paper/2606.00469/) (2026)

## Abstract (arXiv)

We consider the neural ODE perspective of supervised learning and study the impact of the final time T (which may indicate the depth of a corresponding ResNet) in training. For the classical L2--regularized empirical risk minimization problem, whenever the neural ODE dynamics are homogeneous with respect to the parameters, we show that the training error is at most of the order Oleft(frac1Tright). Furthermore, if the loss inducing the empirical risk attains its minimum, the optimal parameters converge to minimal L2--norm parameters which interpolate the dataset. By a natural scaling between T and the regularization hyperparameter λ we obtain the same results when λsearrow0 and T is fixed. This allows us to stipulate generalization properties in the overparametrized regime, now seen from the large depth, neural ODE perspective. To enhance the polynomial decay, inspired by turnpike theory in optimal control, we propose a learning problem with an additional integral regularization term of the neural ODE trajectory over [0,T]. In the setting of ℓp--distance losses, we prove that both the training error and the optimal parameters are at most of the order Oleft(e-μ tright) in any t∈[0,T]. The aforementioned stability estimates are also shown for continuous space-time neural networks, taking the form of nonlinear integro-differential equations. By using a time-dependent moving grid for discretizing the spatial variable, we demonstrate that these equations provide a framework for addressing ResNets with variable widths.