mlsanityData to 7 Oct 2026

Training Stiff Neural Ordinary Differential Equations with Implicit Single-Step Methods

arXiv 2410.05592 ↗First posted 8 Oct 2024math.NA cs.AI cs.CEcited by 11

Not on the map. Why?

Abstract (arXiv)

Stiff systems of ordinary differential equations (ODEs) are pervasive in many science and engineering fields, yet standard neural ODE approaches struggle to learn them. This limitation is the main barrier to the widespread adoption of neural ODEs. In this paper, we propose an approach based on single-step implicit schemes to enable neural ODEs to handle stiffness and demonstrate that our implicit neural ODE method can learn stiff dynamics. This work addresses a key limitation in current neural ODE methods, paving the way for their use in a wider range of scientific problems.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 38 of its 89 references that are in the map. First posted on arXiv 8 Oct 2024.

Builds on

38 of 89 references.

References · 38

  1. 2018Neural Ordinary Differential Equationscited by 3,733
  2. 2020Learning Mesh-Based Simulation with Graph Networkscited by 840
  3. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 603
  4. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  5. 2021Physics-informed neural networks (PINNs) for fluid mechanics: A reviewcited by 470
  6. 2020Scalable Gradients for Stochastic Differential Equationscited by 264
  7. 2022On Neural Differential Equationscited by 264
  8. 2017Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equationscited by 208
  9. 2017Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equationscited by 158
  10. 2020Dissecting Neural ODEscited by 150
  11. 2019Neural Jump Stochastic Differential Equationscited by 147
  12. 2020OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transportcited by 143
26 more
  1. 2020Neural Rough Differential Equations for Long Time Seriescited by 140
  2. 2020SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamicscited by 125
  3. 2019Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discoverycited by 93
  4. 2019ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEscited by 83
  5. 2020Learning Differential Equations that are Easy to Solvecited by 81
  6. 2021Stiff Neural Ordinary Differential Equationscited by 77
  7. 2017Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equationscited by 72
  8. 2020Deep Polynomial Neural Networkscited by 63
  9. 2020STEER: Simple Temporal Regularization For Neural ODEscited by 58
  10. 2022Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systemscited by 50
  11. 2020Bayesian Neural Ordinary Differential Equationscited by 44
  12. 2020Discretize-Optimize vs. Optimize-Discretize for Time-Series Regression and Continuous Normalizing Flowscited by 36
  13. 2018Universal Approximation with Quadratic Deep Networkscited by 29
  14. 2020Autonomous Discovery of Unknown Reaction Pathways from Data by Chemical Reaction Neural Networkcited by 27
  15. 2021Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristicscited by 26
  16. 2022Neural Laplace: Learning diverse classes of differential equations in the Laplace domaincited by 24
  17. 2022Interpretable Polynomial Neural Ordinary Differential Equationscited by 24
  18. 2022Deep Learning and Symbolic Regression for Discovering Parametric Equationscited by 22
  19. 2021Symbolic regression for scientific discovery: an application to wind speed forecastingcited by 16
  20. 2017Internet of Things: An Overviewcited by 14
  21. 2023Toward Physically Plausible Data-Driven Models: A Novel Neural Network Approach to Symbolic Regressioncited by 14
  22. 2024Differential Equations for Continuous-Time Deep Learningcited by 7
  23. 2022Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEscited by 6
  24. 2023Bayesian polynomial neural networks and polynomial neural ordinary differential equationscited by 6
  25. 2023Locally Regularized Neural Differential Equations: Some Black Boxes Were Meant to Remain Closed!cited by 4
  26. 2024Contractivity of neural ODEs: an eigenvalue optimization problemcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 5.

11 papers.

  1. 2024Training Stiff Neural Ordinary Differential Equations with Explicit Exponential Integration Methodscited by 5
  2. 2025SPIN-ODE: Stiff Physics-Informed Neural ODE for Chemical Reaction Rate Estimationcited by 3
  3. 2025The Vanishing Gradient Problem for Stiff Neural Differential Equationscited by 3
  4. 2026Continuous-Time Piecewise-Linear Recurrent Neural Networkscited by 3
  5. 2025Learning solutions of parameterized stiff ODEs using Gaussian processescited by 1
  6. 2025A Weak Penalty Neural ODE for Learning Chaotic Dynamics from Noisy Time Seriescited by 1
  7. 2025Stable CDE Autoencoders with Acuity Regularization for Offline Reinforcement Learning in Sepsis Treatmentcited by 0
  8. 2026A neural operator for predicting vibration frequency response curves from limited datacited by 0
  9. 2026Trajectory-Optimized Time Reparameterization for Learning-Compatible Reduced-Order Modeling of Stiff Dynamical Systemscited by 0
  10. 2026Spatio-Temporal Prediction of Unsteady Airfoil Aerodynamics Using Augmented Graph Neural Ordinary Differential Equations with Exogenous Controlscited by 0
  11. 2026Symbolic Neural ODEs: Learning interpretable models from time-series datacited by 0