mlsanityData to 7 Oct 2026

Training Stiff Neural Ordinary Differential Equations with Explicit Exponential Integration Methods

arXiv 2412.01181 ↗First posted 2 Dec 2024math.NA cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Stiff ordinary differential equations (ODEs) are common in many science and engineering fields, but standard neural ODE approaches struggle to accurately learn these stiff systems, posing a significant barrier to widespread adoption of neural ODEs. In our earlier work, we addressed this challenge by utilizing single-step implicit methods for solving stiff neural ODEs. While effective, these implicit methods are computationally costly and can be complex to implement. This paper expands on our earlier work by exploring explicit exponential integration methods as a more efficient alternative. We evaluate the potential of these explicit methods to handle stiff dynamics in neural ODEs, aiming to enhance their applicability to a broader range of scientific and engineering problems. We found the integrating factor Euler (IF Euler) method to excel in stability and efficiency. While implicit schemes failed to train the stiff Van der Pol oscillator, the IF Euler method succeeded, even with large step sizes. However, IF Euler's first-order accuracy limits its use, leaving the development of higher-order methods for stiff neural ODEs an open research problem.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 41 of its 112 references that are in the map. First posted on arXiv 2 Dec 2024.

Builds on

41 of 112 references.

References · 41

  1. 2018Neural Ordinary Differential Equationscited by 3,733
  2. 2020Learning Mesh-Based Simulation with Graph Networkscited by 840
  3. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 603
  4. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  5. 2021Physics-informed neural networks (PINNs) for fluid mechanics: A reviewcited by 470
  6. 2020Scalable Gradients for Stochastic Differential Equationscited by 264
  7. 2022On Neural Differential Equationscited by 264
  8. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
  9. 2017Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equationscited by 208
  10. 2017Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equationscited by 158
  11. 2020Dissecting Neural ODEscited by 150
  12. 2019Neural Jump Stochastic Differential Equationscited by 147
29 more
  1. 2020OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transportcited by 143
  2. 2020Neural Rough Differential Equations for Long Time Seriescited by 140
  3. 2020SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamicscited by 125
  4. 2019Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discoverycited by 93
  5. 2019ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEscited by 83
  6. 2020Learning Differential Equations that are Easy to Solvecited by 81
  7. 2021Stiff Neural Ordinary Differential Equationscited by 77
  8. 2017Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equationscited by 72
  9. 2020Deep Polynomial Neural Networkscited by 63
  10. 2020STEER: Simple Temporal Regularization For Neural ODEscited by 58
  11. 2022Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systemscited by 50
  12. 2020Bayesian Neural Ordinary Differential Equationscited by 44
  13. 2020Discretize-Optimize vs. Optimize-Discretize for Time-Series Regression and Continuous Normalizing Flowscited by 36
  14. 2018Universal Approximation with Quadratic Deep Networkscited by 29
  15. 2020Autonomous Discovery of Unknown Reaction Pathways from Data by Chemical Reaction Neural Networkcited by 27
  16. 2021Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristicscited by 26
  17. 2020Stable Neural Flowscited by 25
  18. 2022Neural Laplace: Learning diverse classes of differential equations in the Laplace domaincited by 24
  19. 2022Interpretable Polynomial Neural Ordinary Differential Equationscited by 24
  20. 2022Deep Learning and Symbolic Regression for Discovering Parametric Equationscited by 22
  21. 2021Symbolic regression for scientific discovery: an application to wind speed forecastingcited by 16
  22. 2017Internet of Things: An Overviewcited by 14
  23. 2023Toward Physically Plausible Data-Driven Models: A Novel Neural Network Approach to Symbolic Regressioncited by 14
  24. 2024Training Stiff Neural Ordinary Differential Equations with Implicit Single-Step Methodscited by 11
  25. 2024Differential Equations for Continuous-Time Deep Learningcited by 7
  26. 2022Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEscited by 6
  27. 2023Bayesian polynomial neural networks and polynomial neural ordinary differential equationscited by 6
  28. 2023Locally Regularized Neural Differential Equations: Some Black Boxes Were Meant to Remain Closed!cited by 4
  29. 2024Contractivity of neural ODEs: an eigenvalue optimization problemcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Physics-Informed Machine Learning in Biomedical Science and Engineeringcited by 4
  2. 2025SPIN-ODE: Stiff Physics-Informed Neural ODE for Chemical Reaction Rate Estimationcited by 3
  3. 2026SPIKE: Sparse Koopman Regularization for Physics-Informed Neural Networkscited by 0
  4. 2026Trajectory-Optimized Time Reparameterization for Learning-Compatible Reduced-Order Modeling of Stiff Dynamical Systemscited by 0
  5. 2026NEXT: Physics-Informed Neuro-Spectral Exponential Time Differencing Architecturescited by 0