# A Probabilistic State Space Model for Joint Inference from Differential Equations and Data

arXiv: https://arxiv.org/abs/2103.10153 · HTML: https://mlsanity.com/paper/2103.10153/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 53 references that are in the map.
- First posted on arXiv 18 Mar 2021.

## Builds on (most cited first, 12 of 12)

- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro](https://mlsanity.com/paper/1912.11554/) (2019)
- [Elements of Sequential Monte Carlo](https://mlsanity.com/paper/1903.04797/) (2019)
- [Linearly constrained Gaussian processes](https://mlsanity.com/paper/1703.00787/) (2017)
- [A probabilistic model for the numerical solution of initial value problems](https://mlsanity.com/paper/1610.05261/) (2016)
- [Probabilistic Solutions To Ordinary Differential Equations As Non-Linear Bayesian Filtering: A New Perspective](https://mlsanity.com/paper/1810.03440/) (2018)
- [Calibrated Adaptive Probabilistic ODE Solvers](https://mlsanity.com/paper/2012.08202/) (2020)
- [A SIDARTHE Model of COVID-19 Epidemic in Italy](https://mlsanity.com/paper/2003.09861/) (2020)
- [ODIN: ODE-Informed Regression for Parameter and State Inference in Time-Continuous Dynamical Systems](https://mlsanity.com/paper/1902.06278/) (2019)
- [Convergence Rates of Gaussian ODE Filters](https://mlsanity.com/paper/1807.09737/) (2018)
- [Differentiable Likelihoods for Fast Inversion of 'Likelihood-Free' Dynamical Systems](https://mlsanity.com/paper/2002.09301/) (2020)
- [Linear-time inference for Gaussian Processes on one dimension](https://mlsanity.com/paper/2003.05554/) (2020)

## Built on by (most cited first, 21 of 21)

- [All-in-one simulation-based inference](https://mlsanity.com/paper/2404.09636/) (2024)
- [Gaussian Process Priors for Systems of Linear Partial Differential Equations with Constant Coefficients](https://mlsanity.com/paper/2212.14319/) (2022)
- [Constraining Gaussian Processes to Systems of Linear Ordinary Differential Equations](https://mlsanity.com/paper/2208.12515/) (2022)
- [Probabilistic ODE Solutions in Millions of Dimensions](https://mlsanity.com/paper/2110.11812/) (2021)
- [Probabilistic Numerical Method of Lines for Time-Dependent Partial Differential Equations](https://mlsanity.com/paper/2110.11847/) (2021)
- [Fenrir: Physics-Enhanced Regression for Initial Value Problems](https://mlsanity.com/paper/2202.01287/) (2022)
- [ProbNum: Probabilistic Numerics in Python](https://mlsanity.com/paper/2112.02100/) (2021)
- [A Kernel Approach for PDE Discovery and Operator Learning](https://mlsanity.com/paper/2210.08140/) (2022)
- [Physics-Informed Variational State-Space Gaussian Processes](https://mlsanity.com/paper/2409.13876/) (2024)
- [BINDy -- Bayesian identification of nonlinear dynamics with reversible-jump Markov-chain Monte-Carlo](https://mlsanity.com/paper/2408.08062/) (2024)
- [AI for Science: An Emerging Agenda](https://mlsanity.com/paper/2303.04217/) (2023)
- [Probabilistic Exponential Integrators](https://mlsanity.com/paper/2305.14978/) (2023)
- [FNOPE: Simulation-based inference on function spaces with Fourier Neural Operators](https://mlsanity.com/paper/2505.22573/) (2025)
- [Data-Adaptive Probabilistic Likelihood Approximation for Ordinary Differential Equations](https://mlsanity.com/paper/2306.05566/) (2023)
- [Parallel-in-Time Probabilistic Numerical ODE Solvers](https://mlsanity.com/paper/2310.01145/) (2023)
- [Linear-Time Probabilistic Solutions of Boundary Value Problems](https://mlsanity.com/paper/2106.07761/) (2021)
- [Probabilistic Numeric SMC Sampling for Bayesian Nonlinear System Identification in Continuous Time](https://mlsanity.com/paper/2404.12923/) (2024)
- [Adaptive Probabilistic ODE Solvers Without Adaptive Memory Requirements](https://mlsanity.com/paper/2410.10530/) (2024)
- [Bayesian Surrogate Training on Multiple Data Sources: A Hybrid Modeling Strategy](https://mlsanity.com/paper/2412.11875/) (2024)
- [Scalable Bayesian Inference for Nonlinear Conservation Laws](https://mlsanity.com/paper/2605.31127/) (2026)
- [Stable and Scalable Probabilistic Numerical Solvers for Stiff and High-Dimensional ODEs](https://mlsanity.com/paper/2606.08203/) (2026)

## Abstract (arXiv)

Mechanistic models with differential equations are a key component of scientific applications of machine learning. Inference in such models is usually computationally demanding, because it involves repeatedly solving the differential equation. The main problem here is that the numerical solver is hard to combine with standard inference techniques. Recent work in probabilistic numerics has developed a new class of solvers for ordinary differential equations (ODEs) that phrase the solution process directly in terms of Bayesian filtering. We here show that this allows such methods to be combined very directly, with conceptual and numerical ease, with latent force models in the ODE itself. It then becomes possible to perform approximate Bayesian inference on the latent force as well as the ODE solution in a single, linear complexity pass of an extended Kalman filter / smoother - that is, at the cost of computing a single ODE solution. We demonstrate the expressiveness and performance of the algorithm by training, among others, a non-parametric SIRD model on data from the COVID-19 outbreak.