# Multiple shooting for training neural differential equations on time series

arXiv: https://arxiv.org/abs/2109.06786 · HTML: https://mlsanity.com/paper/2109.06786/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 32 references that are in the map.
- First posted on arXiv 14 Sep 2021.

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [Regularisation of Neural Networks by Enforcing Lipschitz Continuity](https://mlsanity.com/paper/1804.04368/) (2018)
- [Training behavior of deep neural network in frequency domain](https://mlsanity.com/paper/1807.01251/) (2018)
- [Fashionable Modelling with Flux](https://mlsanity.com/paper/1811.01457/) (2018)
- [Discretize-Optimize vs. Optimize-Discretize for Time-Series Regression and Continuous Normalizing Flows](https://mlsanity.com/paper/2005.13420/) (2020)
- [DiffEqFlux.jl - A Julia Library for Neural Differential Equations](https://mlsanity.com/paper/1902.02376/) (2019)

## Built on by (most cited first, 17 of 17)

- [Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems](https://mlsanity.com/paper/2306.13867/) (2023)
- [Latent Neural ODEs with Sparse Bayesian Multiple Shooting](https://mlsanity.com/paper/2210.03466/) (2022)
- [Homotopy-based training of NeuralODEs for accurate dynamics discovery](https://mlsanity.com/paper/2210.01407/) (2022)
- [NeuralFMU: Presenting a workflow for integrating hybrid NeuralODEs into real world applications](https://mlsanity.com/paper/2209.03933/) (2022)
- [CONFIDE: Contextual Finite Differences Modelling of PDEs](https://mlsanity.com/paper/2303.15827/) (2023)
- [Effective Latent Differential Equation Models via Attention and Multiple Shooting](https://mlsanity.com/paper/2307.05735/) (2023)
- [Learning Physically Consistent Lagrangian Control Models Without Acceleration Measurements](https://mlsanity.com/paper/2512.03035/) (2025)
- [Learning Time Delay Systems with Neural Ordinary Differential Equations](https://mlsanity.com/paper/2206.14288/) (2022)
- [A condensing approach to multiple shooting neural ordinary differential equation](https://mlsanity.com/paper/2506.00724/) (2025)
- [Stable Robot Motions on Manifolds: Learning Lyapunov-Constrained Neural Manifold ODEs](https://mlsanity.com/paper/2510.05707/) (2025)
- [CODE: A global approach to ODE dynamics learning](https://mlsanity.com/paper/2511.15619/) (2025)
- [Generative prediction of laser-induced rocket ignition with dynamic latent space representations](https://mlsanity.com/paper/2603.07525/) (2026)
- [MPINeuralODE: Multiple-Initial-Condition Physics-Informed Neural ODEs for Globally Consistent Dynamical System Learning](https://mlsanity.com/paper/2605.13305/) (2026)
- [Frequency-Domain Neural ODEs for Modeling Non-Linear Dynamical Systems](https://mlsanity.com/paper/2606.22075/) (2026)
- [Long-Time Trajectory Approximation via SA-NODEs: Model Predictive and Floquet Strategies](https://mlsanity.com/paper/2608.10738/) (2026)
- [TinyUDE: Solver-Free Universal Differential Equations on Microcontrollers via Lie-Taylor Jet Matching](https://mlsanity.com/paper/2609.26972/) (2026)
- [Neural ODEs Meet Concurrent Learning: Stable Online Learning with Lyapunov Guarantees](https://mlsanity.com/paper/2609.32289/) (2026)

## Abstract (arXiv)

Neural differential equations have recently emerged as a flexible data-driven/hybrid approach to model time-series data. This work experimentally demonstrates that if the data contains oscillations, then standard fitting of a neural differential equation may result in a flattened out trajectory that fails to describe the data. We then introduce the multiple shooting method and present successful demonstrations of this method for the fitting of a neural differential equation to two datasets (synthetic and experimental) that the standard approach fails to fit. Constraints introduced by multiple shooting can be satisfied using a penalty or augmented Lagrangian method.