mlsanityData to 7 Oct 2026

Stiff Neural Ordinary Differential Equations

arXiv 2103.15341 ↗First posted 29 Mar 2021math.NA cs.LG cs.NAcited by 77

Not on the map. Why?

Abstract (arXiv)

Neural Ordinary Differential Equations (ODE) are a promising approach to learn dynamic models from time-series data in science and engineering applications. This work aims at learning Neural ODE for stiff systems, which are usually raised from chemical kinetic modeling in chemical and biological systems. We first show the challenges of learning neural ODE in the classical stiff ODE systems of Robertson's problem and propose techniques to mitigate the challenges associated with scale separations in stiff systems. We then present successful demonstrations in stiff systems of Robertson's problem and an air pollution problem. The demonstrations show that the usage of deep networks with rectified activations, proper scaling of the network outputs as well as loss functions, and stabilized gradient calculations are the key techniques enabling the learning of stiff neural ODE. The success of learning stiff neural ODE opens up possibilities of using neural ODEs in applications with widely varying time-scales, like chemical dynamics in energy conversion, environmental engineering, and the life sciences.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). Builds on 14 of its 43 references that are in the map. First posted on arXiv 29 Mar 2021.

Builds on

14 of 43 references.

References · 14

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  5. 2019Deep Equilibrium Modelscited by 603
  6. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 588
  7. 2020Universal Differential Equations for Scientific Machine Learningcited by 345
  8. 2020Understanding and mitigating gradient pathologies in physics-informed neural networkscited by 134
  9. 2019ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEscited by 83
  10. 2020STEER: Simple Temporal Regularization For Neural ODEscited by 58
  11. 2019DiffEqFlux.jl - A Julia Library for Neural Differential Equationscited by 33
  12. 2020Autonomous Discovery of Unknown Reaction Pathways from Data by Chemical Reaction Neural Networkcited by 27
2 more
  1. 2020Universal Battery Performance and Degradation Model for Electric Aircraftcited by 8
  2. 2020Accelerating Simulation of Stiff Nonlinear Systems using Continuous-Time Echo State Networkscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 12; 2023: 12; 2024: 15; 2025: 19; 2026: 12.

77 papers.

Sort:

By area:

  1. 2024KAN-ODEs: Kolmogorov-Arnold Network Ordinary Differential Equations for Learning Dynamical Systems and Hidden Physicscited by 58
  2. 2023Effectively Modeling Time Series with Simple Discrete State Spacescited by 54
  3. 2021Neural Controlled Differential Equations for Online Prediction Taskscited by 46
  4. 2025Physics-Constrained Flow Matching: Sampling Generative Models with Hard Constraintscited by 46
  5. 2022Deep Latent State Space Models for Time-Series Generationcited by 37
  6. 2023Learning stiff chemical kinetics using extended deep neural operatorscited by 30
  7. 2021DAE-PINN: A Physics-Informed Neural Network Model for Simulating Differential-Algebraic Equations with Application to Power Networkscited by 28
  8. 2022Neural Laplace: Learning diverse classes of differential equations in the Laplace domaincited by 24
  9. 2022Interpretable Polynomial Neural Ordinary Differential Equationscited by 24
  10. 2022Differentiable Programming for Earth System Modelingcited by 21
  11. 2025Comprehensive Review of Neural Differential Equations for Time Series Analysiscited by 20
  12. 2021DySMHO: Data-Driven Discovery of Governing Equations for Dynamical Systems via Moving Horizon Optimizationcited by 19
38 more
  1. 2022Parsimonious Physics-Informed Random Projection Neural Networks for Initial-Value Problems of ODEs and index-1 DAEscited by 19
  2. 2023Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraintscited by 15
  3. 2022Residual-based physics-informed transfer learning: A hybrid method for accelerating long-term CFD simulations via deep learningcited by 14
  4. 2022Homotopy-based training of NeuralODEs for accurate dynamics discoverycited by 14
  5. 2024When are dynamical systems learned from time series data statistically accurate?cited by 14
  6. 2021Finite volume method network for acceleration of unsteady computational fluid dynamics: non-reacting and reacting flowscited by 12
  7. 2024Training Stiff Neural Ordinary Differential Equations with Implicit Single-Step Methodscited by 11
  8. 2022On the Forward Invariance of Neural ODEscited by 10
  9. 2024From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systemscited by 9
  10. 2025DUE: A Deep Learning Framework and Library for Modeling Unknown Equationscited by 8
  11. 2024AdjointDEIS: Efficient Gradients for Diffusion Modelscited by 7
  12. 2024Learning Hamiltonian neural Koopman operator and simultaneously sustaining and discovering conservation lawcited by 7
  13. 2023Learning Energy Conserving Dynamics Efficiently with Hamiltonian Gaussian Processescited by 6
  14. 2023Bayesian polynomial neural networks and polynomial neural ordinary differential equationscited by 6
  15. 2024Learning Neural Differential Algebraic Equations via Operator Splittingcited by 6
  16. 2024Training Stiff Neural Ordinary Differential Equations with Explicit Exponential Integration Methodscited by 5
  17. 2025Mitigating mode collapse in normalizing flows by annealing with an adaptive schedule: Application to parameter estimationcited by 5
  18. 2023Locally Regularized Neural Differential Equations: Some Black Boxes Were Meant to Remain Closed!cited by 4
  19. 2025ChemKANs for Combustion Chemistry Modeling and Accelerationcited by 4
  20. 2022A memory-efficient neural ODE framework based on high-level adjoint differentiationcited by 3
  21. 2023Do We Need an Encoder-Decoder to Model Dynamical Systems on Networks?cited by 3
  22. 2023Tasks Makyth Models: Machine Learning Assisted Surrogates for Tipping Pointscited by 3
  23. 2025SPIN-ODE: Stiff Physics-Informed Neural ODE for Chemical Reaction Rate Estimationcited by 3
  24. 2025The Vanishing Gradient Problem for Stiff Neural Differential Equationscited by 3
  25. 2022A Neural Ordinary Differential Equation Model for Visualizing Deep Neural Network Behaviors in Multi-Parametric MRI based Glioma Segmentationcited by 2
  26. 2024Graph Fourier Neural ODEs: Modeling Spatial-temporal Multi-scales in Molecular Dynamicscited by 2
  27. 2024Semi-Implicit Neural Ordinary Differential Equationscited by 2
  28. 2025Greed is Good: A Unifying Perspective on Guided Generationcited by 2
  29. 2025Symbolic Neural Ordinary Differential Equationscited by 2
  30. 2025AMORE: Adaptive Multi-Output Operator Network for Stiff Chemical Kineticscited by 2
  31. 2025Physics-Informed Neural ODEs with Scale-Aware Residuals for Learning Stiff Biophysical Dynamicscited by 2
  32. 2021Incorporating NODE with Pre-trained Neural Differential Operator for Learning Dynamicscited by 1
  33. 2021Numerical Solution of Stiff ODEs with Physics-Informed RPNNscited by 1
  34. 2021Abstract Interpretation of Fixpoint Iterators with Applications to Neural Networkscited by 1
  35. 2022Efficient Climate Simulation via Machine Learning Methodcited by 1
  36. 2023Jacobian-Scaled K-means Clustering for Physics-Informed Segmentation of Reacting Flowscited by 1
  37. 2023A Posteriori Evaluation of a Physics-Constrained Neural Ordinary Differential Equations Approach Coupled with CFD Solver for Modeling Stiff Chemical Kineticscited by 1
  38. 2024FlamePINN-1D: Physics-informed neural networks to solve forward and inverse problems of 1D laminar flamescited by 1

50 most cited of 77 All 77 (JSON)