mlsanityData to 7 Oct 2026

Jacobian Regularization Stabilizes Long-Term Integration of Neural Differential Equations

arXiv 2602.04608 ↗First posted 4 Feb 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Hybrid models and Neural Differential Equations (NDE) are getting increasingly important for the modeling of physical systems, however they often encounter stability and accuracy issues during long-term integration. Training on unrolled trajectories is known to limit these divergences but quickly becomes too expensive due to the need for computing gradients over an iterative process. In this paper, we demonstrate that regularizing the Jacobian of the NDE model via its directional derivatives during training stabilizes long-term integration in the challenging context of short training rollouts. We design two regularizations, one for the case of known dynamics where we can directly derive the directional derivatives of the dynamic and one for the case of unknown dynamics where they are approximated using finite differences. Both methods, while having a far lower cost compared to long rollouts during training, are successful in improving the stability of long-term simulations for several ordinary and partial differential equations, opening up the door to training NDE methods for long-term integration of large scale systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 28 references that are in the map. First posted on arXiv 4 Feb 2026.

Builds on

19 of 28 references.

References · 19

  1. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  2. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  3. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  4. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
  5. 2018Sensitivity and Generalization in Neural Networks: an Empirical Studycited by 187
  6. 2018Semidefinite relaxations for certifying robustness to adversarial examplescited by 178
  7. 2020Exactly Computing the Local Lipschitz Constant of ReLU Networkscited by 102
  8. 2020Training robust neural networks using Lipschitz boundscited by 98
  9. 2020Augmenting Physical Models with Deep Networks for Complex Dynamics Forecastingcited by 93
  10. 2021Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attackscited by 91
  11. 2021Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Boundscited by 64
  12. 2022Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systemscited by 50
7 more
  1. 2023Direct Parameterization of Lipschitz-Bounded Deep Networkscited by 49
  2. 2023Hutchinson Trace Estimation for High-Dimensional and High-Order Physics-Informed Neural Networkscited by 23
  3. 2022A Quantitative Geometric Approach to Neural-Network Smoothnesscited by 16
  4. 2023Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraintscited by 15
  5. 2023Stability-Informed Initialization of Neural Ordinary Differential Equationscited by 5
  6. 2025Semi-Explicit Neural DAEs: Learning Long-Horizon Dynamical Systems with Algebraic Constraintscited by 4
  7. 2021A posteriori learning of quasi-geostrophic turbulence parametrization: an experiment on integration stepscited by 1

Built on by

Not cited yet.