# Experimental study of Neural ODE training with adaptive solver for dynamical systems modeling

arXiv: https://arxiv.org/abs/2211.06972 · HTML: https://mlsanity.com/paper/2211.06972/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 13 references that are in the map.
- First posted on arXiv 13 Nov 2022.

## Builds on (most cited first, 7 of 7)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations](https://mlsanity.com/paper/1710.10121/) (2017)
- [Chaos as an interpretable benchmark for forecasting and data-driven modelling](https://mlsanity.com/paper/2110.05266/) (2021)
- [Tree-Structured Recurrent Switching Linear Dynamical Systems for Multi-Scale Modeling](https://mlsanity.com/paper/1811.12386/) (2018)

## Built on by (most cited first, 1 of 1)

- [A Weak Penalty Neural ODE for Learning Chaotic Dynamics from Noisy Time Series](https://mlsanity.com/paper/2511.06609/) (2025)

## Abstract (arXiv)

Neural Ordinary Differential Equations (ODEs) was recently introduced as a new family of neural network models, which relies on black-box ODE solvers for inference and training. Some ODE solvers called adaptive can adapt their evaluation strategy depending on the complexity of the problem at hand, opening great perspectives in machine learning. However, this paper describes a simple set of experiments to show why adaptive solvers cannot be seamlessly leveraged as a black-box for dynamical systems modelling. By taking the Lorenz'63 system as a showcase, we show that a naive application of the Fehlberg's method does not yield the expected results. Moreover, a simple workaround is proposed that assumes a tighter interaction between the solver and the training strategy. The code is available on github: https://github.com/Allauzen/adaptive-step-size-neural-ode