mlsanityData to 7 Oct 2026

Stability-Informed Initialization of Neural Ordinary Differential Equations

arXiv 2311.15890 ↗First posted 27 Nov 2023cs.LG cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

This paper addresses the training of Neural Ordinary Differential Equations (neural ODEs), and in particular explores the interplay between numerical integration techniques, stability regions, step size, and initialization techniques. It is shown how the choice of integration technique implicitly regularizes the learned model, and how the solver's corresponding stability region affects training and prediction performance. From this analysis, a stability-informed parameter initialization technique is introduced. The effectiveness of the initialization method is displayed across several learning benchmarks and industrial applications.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 40 of its 67 references that are in the map. First posted on arXiv 27 Nov 2023.

Builds on

40 of 67 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2022Flow Matching for Generative Modelingcited by 5,426
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2018Neural Ordinary Differential Equationscited by 3,733
  7. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  8. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  9. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  10. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  11. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  12. 2018On the Spectral Bias of Neural Networkscited by 1,321
28 more
  1. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  2. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
  3. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  4. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
  5. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  6. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  7. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
  8. 2022It's Raw! Audio Generation with State-Space Modelscited by 171
  9. 2020Dissecting Neural ODEscited by 150
  10. 2020Frequency Bias in Neural Networks for Input of Non-Uniform Densitycited by 140
  11. 2019Hamiltonian Generative Networkscited by 136
  12. 2015A Simple Way to Initialize Recurrent Networks of Rectified Linear Unitscited by 130
  13. 2019ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEscited by 83
  14. 2019On Robustness of Neural Ordinary Differential Equationscited by 83
  15. 2020Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODEcited by 76
  16. 2020Learning Continuous System Dynamics from Irregularly-Sampled Partial Observationscited by 60
  17. 2019Approximation Capabilities of Neural ODEs and Invertible Residual Networkscited by 56
  18. 2021Heavy Ball Neural Ordinary Differential Equationscited by 42
  19. 2020Continuous-in-Depth Neural Networkscited by 40
  20. 2020Benchmarking Energy-Conserving Neural Networks for Learning Dynamics from Datacited by 40
  21. 2022Learning continuous models for continuous physicscited by 28
  22. 2021Which priors matter? Benchmarking models for learning latent dynamicscited by 26
  23. 2022On Numerical Integration in Neural Ordinary Differential Equationscited by 21
  24. 2020ResNet After All? Neural ODEs and Their Numerical Solutioncited by 20
  25. 2023MTP-GO: Graph-Based Probabilistic Multi-Agent Trajectory Prediction with Neural ODEscited by 19
  26. 2020Gradient Amplification: An efficient way to train deep neural networkscited by 15
  27. 2020Towards Understanding Normalization in Neural ODEscited by 7
  28. 2023Analysis of Numerical Integration in RNN-Based Residuals for Fault Diagnosis of Dynamic Systemscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026Lyapunov Probes for Hallucination Detection in Large Foundation Modelscited by 3
  2. 2025Learning the Simplest Neural ODEcited by 0
  3. 2026Jacobian Regularization Stabilizes Long-Term Integration of Neural Differential Equationscited by 0
  4. 2026A neural operator for predicting vibration frequency response curves from limited datacited by 0
  5. 2026Latent Laplace Diffusion for Irregular Multivariate Time Seriescited by 0