mlsanityData to 7 Oct 2026

On Numerical Integration in Neural Ordinary Differential Equations

arXiv 2206.07335 ↗First posted 15 Jun 2022cs.LG cs.NA math.NAcited by 21

Not on the map. Why?

Abstract (arXiv)

The combination of ordinary differential equations and neural networks, i.e., neural ordinary differential equations (Neural ODE), has been widely studied from various angles. However, deciphering the numerical integration in Neural ODE is still an open challenge, as many researches demonstrated that numerical integration significantly affects the performance of the model. In this paper, we propose the inverse modified differential equations (IMDE) to clarify the influence of numerical integration on training Neural ODE models. IMDE is determined by the learning task and the employed ODE solver. It is shown that training a Neural ODE model actually returns a close approximation of the IMDE, rather than the true ODE. With the help of IMDE, we deduce that (i) the discrepancy between the learned model and the true ODE is bounded by the sum of discretization error and learning loss; (ii) Neural ODE using non-symplectic numerical integration fail to learn conservation laws theoretically. Several experiments are performed to numerically verify our theoretical analysis.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 30 of its 57 references that are in the map. First posted on arXiv 15 Jun 2022.

Builds on

30 of 57 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018Neural Ordinary Differential Equationscited by 3,733
  3. 2019Hamiltonian Neural Networkscited by 702
  4. 2019Augmented Neural ODEscited by 396
  5. 2017Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equationscited by 203
  6. 2018Training behavior of deep neural network in frequency domaincited by 174
  7. 2020Dissecting Neural ODEscited by 150
  8. 2019Neural Jump Stochastic Differential Equationscited by 147
  9. 2018Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systemscited by 88
  10. 2019On Robustness of Neural Ordinary Differential Equationscited by 83
  11. 2017Maximum Principle Based Algorithms for Deep Learningcited by 77
  12. 2020Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODEcited by 76
18 more
  1. 2018A Mean-Field Optimal Control Formulation of Deep Learningcited by 67
  2. 2021Neural ODE Processescited by 57
  3. 2019Approximation Capabilities of Neural ODEs and Invertible Residual Networkscited by 56
  4. 2019Theory of the Frequency Principle for General Deep Neural Networkscited by 45
  5. 2021Heavy Ball Neural Ordinary Differential Equationscited by 42
  6. 2021Data-driven Prediction of General Hamiltonian Dynamics via Learning Exactly-Symplectic Mapscited by 41
  7. 2020Continuous-in-Depth Neural Networkscited by 40
  8. 2019ODE2VAE: Deep generative second order ODEs with Bayesian neural networkscited by 36
  9. 2020OnsagerNet: Learning Stable and Interpretable Dynamics using a Generalized Onsager Principlecited by 35
  10. 2020Hypersolvers: Toward Fast Continuous-Depth Modelscited by 33
  11. 2022Learning continuous models for continuous physicscited by 28
  12. 2021Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristicscited by 26
  13. 2020On dissipative symplectic integration with applications to gradient-based optimizationcited by 23
  14. 2020ResNet After All? Neural ODEs and Their Numerical Solutioncited by 20
  15. 2020Time-Reversal Symmetric ODE Networkcited by 19
  16. 2019Discovery of Dynamics Using Linear Multistep Methodscited by 16
  17. 2021The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimationcited by 14
  18. 2020Towards Understanding Normalization in Neural ODEscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 9; 2024: 1; 2025: 6; 2026: 4.

21 papers.

Sort:

By area:

  1. 2023SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biasescited by 27
  2. 2023Bayesian Gaussian Process ODEs via Double Normalizing Flowscited by 7
  3. 2023Implementation and (Inverse Modified) Error Analysis for implicitly-templated ODE-netscited by 6
  4. 2023Stability-Informed Initialization of Neural Ordinary Differential Equationscited by 5
  5. 2022SeqLink: A Robust Neural-ODE Architecture for Modelling Partially Observed Time Seriescited by 4
  6. 2023Some of the variables, some of the parameters, some of the times, with some physics known: Identification with partial informationcited by 4
  7. 2023Learning Hamiltonian Systems with Mono-Implicit Runge-Kutta Methodscited by 3
  8. 2025Numerical and statistical analysis of NeuralODE with Runge-Kutta time integrationcited by 3
  9. 2025Diffusion Secant Alignment for Score-Based Density Ratio Estimationcited by 3
  10. 2023Analysis of Numerical Integration in RNN-Based Residuals for Fault Diagnosis of Dynamic Systemscited by 2
  11. 2023Pseudo-Hamiltonian system identificationcited by 1
  12. 2024A Unified Framework for Neural Computation and Learning Over Timecited by 1
9 more
  1. 2025Artifacts of Numerical Integration in Learning Dynamical Systemscited by 1
  2. 2026Controlled oscillation modeling using port-Hamiltonian neural networkscited by 1
  3. 2023Time-vectorized numerical integration for systems of ODEscited by 0
  4. 2025Federated Spectral Graph Transformers Meet Neural Ordinary Differential Equations for Non-IID Graphscited by 0
  5. 2025Interpretability and Generalization Bounds for Learning Spatial Physicscited by 0
  6. 2025FNODE: Flow-Matching for data-driven simulation of constrained multibody systemscited by 0
  7. 2026Sharpen Your Flow: Sharpness-Aware Sampling for Flow Matchingcited by 0
  8. 2026RTS Smoother-Guided Learning of Physics-Based Neural Differential Modelscited by 0
  9. 2026Variational Streaming Flow: Probabilistic Forecasting in Physical Timecited by 0