# Variational multiple shooting for Bayesian ODEs with Gaussian processes

arXiv: https://arxiv.org/abs/2106.10905 · HTML: https://mlsanity.com/paper/2106.10905/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 68 references that are in the map.
- First posted on arXiv 21 Jun 2021.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [Latent ODEs for Irregularly-Sampled Time Series](https://mlsanity.com/paper/1907.03907/) (2019)
- [Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit](https://mlsanity.com/paper/1905.09883/) (2019)
- [Efficiently Sampling Functions from Gaussian Process Posteriors](https://mlsanity.com/paper/2002.09309/) (2020)
- [Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODE](https://mlsanity.com/paper/2006.02493/) (2020)
- [Probabilistic Recurrent State-Space Models](https://mlsanity.com/paper/1801.10395/) (2018)
- [Learning unknown ODE models with Gaussian processes](https://mlsanity.com/paper/1803.04303/) (2018)
- [Bayesian Neural Ordinary Differential Equations](https://mlsanity.com/paper/2012.07244/) (2020)
- [MCMC for Variationally Sparse Gaussian Processes](https://mlsanity.com/paper/1506.04000/) (2015)
- [Variational Gaussian Process State-Space Models](https://mlsanity.com/paper/1406.4905/) (2014)
- [MALI: A memory efficient and reverse accurate integrator for Neural ODEs](https://mlsanity.com/paper/2102.04668/) (2021)
- [Identification of Gaussian Process State Space Models](https://mlsanity.com/paper/1705.10888/) (2017)
- [Learning interpretable continuous-time models of latent stochastic dynamical systems](https://mlsanity.com/paper/1902.04420/) (2019)
- [Random Fourier Features for Operator-Valued Kernels](https://mlsanity.com/paper/1605.02536/) (2016)
- [Overcoming Mean-Field Approximations in Recurrent Gaussian Process Models](https://mlsanity.com/paper/1906.05828/) (2019)
- [Differentiable Multiple Shooting Layers](https://mlsanity.com/paper/2106.03885/) (2021)
- [ODIN: ODE-Informed Regression for Parameter and State Inference in Time-Continuous Dynamical Systems](https://mlsanity.com/paper/1902.06278/) (2019)
- [An Ode to an ODE](https://mlsanity.com/paper/2006.11421/) (2020)
- [Gaussian processes meet NeuralODEs: A Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data](https://mlsanity.com/paper/2103.03385/) (2021)
- [Learning ODE Models with Qualitative Structure Using Gaussian Processes](https://mlsanity.com/paper/2011.05364/) (2020)
- [Structure-preserving Gaussian Process Dynamics](https://mlsanity.com/paper/2102.01606/) (2021)
- [Monotonic Gaussian Process Flow](https://mlsanity.com/paper/1905.12930/) (2019)
- [Learning nonparametric differential equations with operator-valued kernels and gradient matching](https://mlsanity.com/paper/1411.5172/) (2014)
- [Stochastic Differential Equations with Variational Wishart Diffusions](https://mlsanity.com/paper/2006.14895/) (2020)

## Built on by (most cited first, 15 of 15)

- [Latent Neural ODEs with Sparse Bayesian Multiple Shooting](https://mlsanity.com/paper/2210.03466/) (2022)
- [Modulated Neural ODEs](https://mlsanity.com/paper/2302.13262/) (2023)
- [Free-Form Variational Inference for Gaussian Process State-Space Models](https://mlsanity.com/paper/2302.09921/) (2023)
- [Bayesian Gaussian Process ODEs via Double Normalizing Flows](https://mlsanity.com/paper/2309.09222/) (2023)
- [Learning Energy Conserving Dynamics Efficiently with Hamiltonian Gaussian Processes](https://mlsanity.com/paper/2303.01925/) (2023)
- [Learning Space-Time Continuous Neural PDEs from Partially Observed States](https://mlsanity.com/paper/2307.04110/) (2023)
- [System-Aware Neural ODE Processes for Few-Shot Bayesian Optimization](https://mlsanity.com/paper/2406.02352/) (2024)
- [Foundation Inference Models for Ordinary Differential Equations](https://mlsanity.com/paper/2602.08733/) (2026)
- [The Past Does Matter: Correlation of Subsequent States in Trajectory Predictions of Gaussian Process Models](https://mlsanity.com/paper/2211.11103/) (2022)
- [Variational Gaussian Processes For Linear Inverse Problems](https://mlsanity.com/paper/2311.00663/) (2023)
- [Counterfactual-based Root Cause Analysis for Dynamical Systems](https://mlsanity.com/paper/2406.08106/) (2024)
- [Causal Structure Learning for Dynamical Systems with Theoretical Score Analysis](https://mlsanity.com/paper/2512.14361/) (2025)
- [CODE: A global approach to ODE dynamics learning](https://mlsanity.com/paper/2511.15619/) (2025)
- [Bayesian Nonparametric Mixed-Effect ODEs with Gaussian Processes](https://mlsanity.com/paper/2605.13088/) (2026)
- [Function-Space Priors for Bayesian Neural ODEs with Application to Vessel Trajectory Prediction](https://mlsanity.com/paper/2606.06351/) (2026)

## Abstract (arXiv)

Recent machine learning advances have proposed black-box estimation of unknown continuous-time system dynamics directly from data. However, earlier works are based on approximative ODE solutions or point estimates. We propose a novel Bayesian nonparametric model that uses Gaussian processes to infer posteriors of unknown ODE systems directly from data. We derive sparse variational inference with decoupled functional sampling to represent vector field posteriors. We also introduce a probabilistic shooting augmentation to enable efficient inference from arbitrarily long trajectories. The method demonstrates the benefit of computing vector field posteriors, with predictive uncertainty scores outperforming alternative methods on multiple ODE learning tasks.