# Training Stiff Neural Ordinary Differential Equations with Explicit Exponential Integration Methods

arXiv: https://arxiv.org/abs/2412.01181 · HTML: https://mlsanity.com/paper/2412.01181/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 112 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Builds on (most cited first, 30 of 41)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [On Neural Differential Equations](https://mlsanity.com/paper/2202.02435/) (2022)
- [How to train your neural ODE: the world of Jacobian and kinetic regularization](https://mlsanity.com/paper/2002.02798/) (2020)
- [Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1708.00588/) (2017)
- [Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10566/) (2017)
- [Dissecting Neural ODEs](https://mlsanity.com/paper/2002.08071/) (2020)
- [Neural Jump Stochastic Differential Equations](https://mlsanity.com/paper/1905.10403/) (2019)
- [OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport](https://mlsanity.com/paper/2006.00104/) (2020)
- [Neural Rough Differential Equations for Long Time Series](https://mlsanity.com/paper/2009.08295/) (2020)
- [SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamics](https://mlsanity.com/paper/2004.02322/) (2020)
- [Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery](https://mlsanity.com/paper/1912.04825/) (2019)
- [ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs](https://mlsanity.com/paper/1902.10298/) (2019)
- [Learning Differential Equations that are Easy to Solve](https://mlsanity.com/paper/2007.04504/) (2020)
- [Stiff Neural Ordinary Differential Equations](https://mlsanity.com/paper/2103.15341/) (2021)
- [Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations](https://mlsanity.com/paper/1703.10230/) (2017)
- [Deep Polynomial Neural Networks](https://mlsanity.com/paper/2006.13026/) (2020)
- [STEER: Simple Temporal Regularization For Neural ODEs](https://mlsanity.com/paper/2006.10711/) (2020)
- [Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systems](https://mlsanity.com/paper/2203.15706/) (2022)
- [Bayesian Neural Ordinary Differential Equations](https://mlsanity.com/paper/2012.07244/) (2020)
- [Discretize-Optimize vs. Optimize-Discretize for Time-Series Regression and Continuous Normalizing Flows](https://mlsanity.com/paper/2005.13420/) (2020)
- [Universal Approximation with Quadratic Deep Networks](https://mlsanity.com/paper/1808.00098/) (2018)
- [Autonomous Discovery of Unknown Reaction Pathways from Data by Chemical Reaction Neural Network](https://mlsanity.com/paper/2002.09062/) (2020)
- [Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristics](https://mlsanity.com/paper/2105.03918/) (2021)
- [Stable Neural Flows](https://mlsanity.com/paper/2003.08063/) (2020)
- [Neural Laplace: Learning diverse classes of differential equations in the Laplace domain](https://mlsanity.com/paper/2206.04843/) (2022)

## Built on by (most cited first, 5 of 5)

- [Physics-Informed Machine Learning in Biomedical Science and Engineering](https://mlsanity.com/paper/2510.05433/) (2025)
- [SPIN-ODE: Stiff Physics-Informed Neural ODE for Chemical Reaction Rate Estimation](https://mlsanity.com/paper/2505.05625/) (2025)
- [SPIKE: Sparse Koopman Regularization for Physics-Informed Neural Networks](https://mlsanity.com/paper/2601.10282/) (2026)
- [Trajectory-Optimized Time Reparameterization for Learning-Compatible Reduced-Order Modeling of Stiff Dynamical Systems](https://mlsanity.com/paper/2603.16583/) (2026)
- [NEXT: Physics-Informed Neuro-Spectral Exponential Time Differencing Architectures](https://mlsanity.com/paper/2609.31539/) (2026)

## Abstract (arXiv)

Stiff ordinary differential equations (ODEs) are common in many science and engineering fields, but standard neural ODE approaches struggle to accurately learn these stiff systems, posing a significant barrier to widespread adoption of neural ODEs. In our earlier work, we addressed this challenge by utilizing single-step implicit methods for solving stiff neural ODEs. While effective, these implicit methods are computationally costly and can be complex to implement. This paper expands on our earlier work by exploring explicit exponential integration methods as a more efficient alternative. We evaluate the potential of these explicit methods to handle stiff dynamics in neural ODEs, aiming to enhance their applicability to a broader range of scientific and engineering problems. We found the integrating factor Euler (IF Euler) method to excel in stability and efficiency. While implicit schemes failed to train the stiff Van der Pol oscillator, the IF Euler method succeeded, even with large step sizes. However, IF Euler's first-order accuracy limits its use, leaving the development of higher-order methods for stiff neural ODEs an open research problem.