# Variational Gaussian Process Diffusion Processes

arXiv: https://arxiv.org/abs/2306.02066 · HTML: https://mlsanity.com/paper/2306.02066/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 68 references that are in the map.
- First posted on arXiv 3 Jun 2023.

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation](https://mlsanity.com/paper/2107.03502/) (2021)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [Solving Schr"odinger Bridges via Maximum Likelihood](https://mlsanity.com/paper/2106.02081/) (2021)
- [Efficient and Accurate Gradients for Neural SDEs](https://mlsanity.com/paper/2105.13493/) (2021)
- [Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models](https://mlsanity.com/paper/1703.04265/) (2017)
- [Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations](https://mlsanity.com/paper/1906.02914/) (2019)
- [Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Models](https://mlsanity.com/paper/1807.04489/) (2018)
- [Learning interpretable continuous-time models of latent stochastic dynamical systems](https://mlsanity.com/paper/1902.04420/) (2019)
- [Black-box Variational Inference for Stochastic Differential Equations](https://mlsanity.com/paper/1802.03335/) (2018)
- [Learning Stochastic Differential Equations With Gaussian Processes Without Gradient Matching](https://mlsanity.com/paper/1807.05748/) (2018)
- [Particle Gibbs with Ancestor Sampling](https://mlsanity.com/paper/1401.0604/) (2014)
- [Fast Variational Learning in State-Space Gaussian Process Models](https://mlsanity.com/paper/2007.04731/) (2020)
- [Dual Parameterization of Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2111.03412/) (2021)
- [Bayes-Newton Methods for Approximate Bayesian Inference with PSD Guarantees](https://mlsanity.com/paper/2111.01721/) (2021)
- [Non-parametric Estimation of Stochastic Differential Equations with Sparse Gaussian Processes](https://mlsanity.com/paper/1704.04375/) (2017)
- [Sparse Algorithms for Markovian Gaussian Processes](https://mlsanity.com/paper/2103.10710/) (2021)
- [Variational Inference for Continuous-Time Switching Dynamical Systems](https://mlsanity.com/paper/2109.14492/) (2021)
- [Moment-Based Variational Inference for Stochastic Differential Equations](https://mlsanity.com/paper/2103.00988/) (2021)
- [Expectation propagation for continuous time stochastic processes](https://mlsanity.com/paper/1512.06098/) (2015)
- [Nonlinear state space smoothing using the conditional particle filter](https://mlsanity.com/paper/1502.03697/) (2015)
- [Gaussian process classification using posterior linearisation](https://mlsanity.com/paper/1809.04967/) (2018)

## Built on by (most cited first, 11 of 11)

- [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023)
- [SDE Matching: Scalable and Simulation-Free Training of Latent Stochastic Differential Equations](https://mlsanity.com/paper/2502.02472/) (2025)
- [Modeling Latent Neural Dynamics with Gaussian Process Switching Linear Dynamical Systems](https://mlsanity.com/paper/2408.03330/) (2024)
- [In-Context Learning of Stochastic Differential Equations with Foundation Inference Models](https://mlsanity.com/paper/2502.19049/) (2025)
- [SING: SDE Inference via Natural Gradients](https://mlsanity.com/paper/2506.17796/) (2025)
- [Topological Schr"odinger Bridge Matching](https://mlsanity.com/paper/2504.04799/) (2025)
- [Foundation Inference Models for Ordinary Differential Equations](https://mlsanity.com/paper/2602.08733/) (2026)
- [Towards Fast Coarse-graining and Equation Discovery with Foundation Inference Models](https://mlsanity.com/paper/2510.12618/) (2025)
- [Learning to Approximate Particle Smoothing Trajectories via Diffusion Generative Models](https://mlsanity.com/paper/2406.00561/) (2024)
- [Recursive Entropic Variational Inference for Nonlinear State-Space Models](https://mlsanity.com/paper/2511.15409/) (2025)
- [Gaussian Flow Dynamics: Simulation-Free Neural SDE Learning Beyond One-Time Marginals](https://mlsanity.com/paper/2610.04390/) (2026)

## Abstract (arXiv)

Diffusion processes are a class of stochastic differential equations (SDEs) providing a rich family of expressive models that arise naturally in dynamic modelling tasks. Probabilistic inference and learning under generative models with latent processes endowed with a non-linear diffusion process prior are intractable problems. We build upon work within variational inference, approximating the posterior process as a linear diffusion process, and point out pathologies in the approach. We propose an alternative parameterization of the Gaussian variational process using a site-based exponential family description. This allows us to trade a slow inference algorithm with fixed-point iterations for a fast algorithm for convex optimization akin to natural gradient descent, which also provides a better objective for learning model parameters.