# Scalable Inference in SDEs by Direct Matching of the Fokker-Planck-Kolmogorov Equation

arXiv: https://arxiv.org/abs/2110.15739 · HTML: https://mlsanity.com/paper/2110.15739/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 53 references that are in the map.
- First posted on arXiv 29 Oct 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [Scalable Variational Gaussian Process Classification](https://mlsanity.com/paper/1411.2005/) (2014)
- [Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit](https://mlsanity.com/paper/1905.09883/) (2019)
- [Neural SDEs as Infinite-Dimensional GANs](https://mlsanity.com/paper/2102.03657/) (2021)
- [GPflow: A Gaussian process library using TensorFlow](https://mlsanity.com/paper/1610.08733/) (2016)
- [Gaussian Process Prior Variational Autoencoders](https://mlsanity.com/paper/1810.11738/) (2018)
- [Learning unknown ODE models with Gaussian processes](https://mlsanity.com/paper/1803.04303/) (2018)
- [Variational Gaussian Process State-Space Models](https://mlsanity.com/paper/1406.4905/) (2014)
- [ODE2VAE: Deep generative second order ODEs with Bayesian neural networks](https://mlsanity.com/paper/1905.10994/) (2019)
- [Learning Stochastic Differential Equations With Gaussian Processes Without Gradient Matching](https://mlsanity.com/paper/1807.05748/) (2018)
- [Deep Temporal Sigmoid Belief Networks for Sequence Modeling](https://mlsanity.com/paper/1509.07087/) (2015)
- [Deep learning with differential Gaussian process flows](https://mlsanity.com/paper/1810.04066/) (2018)
- [Adaptive Path-Integral Autoencoder: Representation Learning and Planning for Dynamical Systems](https://mlsanity.com/paper/1807.02128/) (2018)

## Built on by (most cited first, 14 of 14)

- [Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Series](https://mlsanity.com/paper/2301.11308/) (2023)
- [Self-Consistency of the Fokker-Planck Equation](https://mlsanity.com/paper/2206.00860/) (2022)
- [Modulated Neural ODEs](https://mlsanity.com/paper/2302.13262/) (2023)
- [DynGMA: a robust approach for learning stochastic differential equations from data](https://mlsanity.com/paper/2402.14475/) (2024)
- [Bayesian Gaussian Process ODEs via Double Normalizing Flows](https://mlsanity.com/paper/2309.09222/) (2023)
- [Learning Interacting Dynamical Systems with Latent Gaussian Process ODEs](https://mlsanity.com/paper/2205.11894/) (2022)
- [Cheap and Deterministic Inference for Deep State-Space Models of Interacting Dynamical Systems](https://mlsanity.com/paper/2305.01773/) (2023)
- [Neural Stochastic Flows: Solver-Free Modelling and Inference for SDE Solutions](https://mlsanity.com/paper/2510.25769/) (2025)
- [Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equations](https://mlsanity.com/paper/2408.06069/) (2024)
- [Latent Space Energy-based Neural ODEs](https://mlsanity.com/paper/2409.03845/) (2024)
- [Foundation Inference Models for Ordinary Differential Equations](https://mlsanity.com/paper/2602.08733/) (2026)
- [Sampling-Free Probabilistic Deep State-Space Models](https://mlsanity.com/paper/2309.08256/) (2023)
- [Learning Stochastic Hamiltonian Systems via Stochastic Generating Function Neural Network](https://mlsanity.com/paper/2507.14467/) (2025)
- [Learning to Approximate Particle Smoothing Trajectories via Diffusion Generative Models](https://mlsanity.com/paper/2406.00561/) (2024)

## Abstract (arXiv)

Simulation-based techniques such as variants of stochastic Runge-Kutta are the de facto approach for inference with stochastic differential equations (SDEs) in machine learning. These methods are general-purpose and used with parametric and non-parametric models, and neural SDEs. Stochastic Runge-Kutta relies on the use of sampling schemes that can be inefficient in high dimensions. We address this issue by revisiting the classical SDE literature and derive direct approximations to the (typically intractable) Fokker-Planck-Kolmogorov equation by matching moments. We show how this workflow is fast, scales to high-dimensional latent spaces, and is applicable to scarce-data applications, where a non-parametric SDE with a driving Gaussian process velocity field specifies the model.