# Free-Form Variational Inference for Gaussian Process State-Space Models

arXiv: https://arxiv.org/abs/2302.09921 · HTML: https://mlsanity.com/paper/2302.09921/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 51 references that are in the map.
- First posted on arXiv 20 Feb 2023.

## Builds on (most cited first, 25 of 25)

- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)](https://mlsanity.com/paper/1503.01057/) (2015)
- [Doubly Stochastic Variational Inference for Deep Gaussian Processes](https://mlsanity.com/paper/1705.08933/) (2017)
- [Efficiently Sampling Functions from Gaussian Process Posteriors](https://mlsanity.com/paper/2002.09309/) (2020)
- [Variational Fourier features for Gaussian processes](https://mlsanity.com/paper/1611.06740/) (2016)
- [Understanding Probabilistic Sparse Gaussian Process Approximations](https://mlsanity.com/paper/1606.04820/) (2016)
- [Probabilistic Recurrent State-Space Models](https://mlsanity.com/paper/1801.10395/) (2018)
- [The Signature Kernel is the solution of a Goursat PDE](https://mlsanity.com/paper/2006.14794/) (2020)
- [Learning unknown ODE models with Gaussian processes](https://mlsanity.com/paper/1803.04303/) (2018)
- [MCMC for Variationally Sparse Gaussian Processes](https://mlsanity.com/paper/1506.04000/) (2015)
- [Variational Gaussian Process State-Space Models](https://mlsanity.com/paper/1406.4905/) (2014)
- [Random Feature Expansions for Deep Gaussian Processes](https://mlsanity.com/paper/1610.04386/) (2016)
- [Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1806.05490/) (2018)
- [Identification of Gaussian Process State Space Models](https://mlsanity.com/paper/1705.10888/) (2017)
- [Learning interpretable continuous-time models of latent stochastic dynamical systems](https://mlsanity.com/paper/1902.04420/) (2019)
- [Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC](https://mlsanity.com/paper/1306.2861/) (2013)
- [SigGPDE: Scaling Sparse Gaussian Processes on Sequential Data](https://mlsanity.com/paper/2105.04211/) (2021)
- [Overcoming Mean-Field Approximations in Recurrent Gaussian Process Models](https://mlsanity.com/paper/1906.05828/) (2019)
- [Computationally Efficient Bayesian Learning of Gaussian Process State Space Models](https://mlsanity.com/paper/1506.02267/) (2015)
- [Implicit Posterior Variational Inference for Deep Gaussian Processes](https://mlsanity.com/paper/1910.11998/) (2019)
- [Variational multiple shooting for Bayesian ODEs with Gaussian processes](https://mlsanity.com/paper/2106.10905/) (2021)
- [State Space Gaussian Processes with Non-Gaussian Likelihood](https://mlsanity.com/paper/1802.04846/) (2018)
- [Variational Orthogonal Features](https://mlsanity.com/paper/2006.13170/) (2020)
- [Efficient Inference in Multi-task Cox Process Models](https://mlsanity.com/paper/1805.09781/) (2018)

## Built on by (most cited first, 7 of 7)

- [Koopman-Equivariant Gaussian Processes](https://mlsanity.com/paper/2502.06645/) (2025)
- [Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference](https://mlsanity.com/paper/2312.05910/) (2023)
- [FedSI: Federated Subnetwork Inference for Efficient Uncertainty Quantification](https://mlsanity.com/paper/2404.15657/) (2024)
- [Towards Efficient Modeling and Inference in Multi-Dimensional Gaussian Process State-Space Models](https://mlsanity.com/paper/2309.01074/) (2023)
- [Efficient Transformed Gaussian Process State-Space Models for Non-Stationary High-Dimensional Dynamical Systems](https://mlsanity.com/paper/2503.18309/) (2025)
- [Interpretable neural network system identification method for two families of second-order systems based on characteristic curves](https://mlsanity.com/paper/2509.10632/) (2025)
- [Recursive Gaussian Process State Space Model](https://mlsanity.com/paper/2411.14679/) (2024)

## Abstract (arXiv)

Gaussian process state-space models (GPSSMs) provide a principled and flexible approach to modeling the dynamics of a latent state, which is observed at discrete-time points via a likelihood model. However, inference in GPSSMs is computationally and statistically challenging due to the large number of latent variables in the model and the strong temporal dependencies between them. In this paper, we propose a new method for inference in Bayesian GPSSMs, which overcomes the drawbacks of previous approaches, namely over-simplified assumptions, and high computational requirements. Our method is based on free-form variational inference via stochastic gradient Hamiltonian Monte Carlo within the inducing-variable formalism. Furthermore, by exploiting our proposed variational distribution, we provide a collapsed extension of our method where the inducing variables are marginalized analytically. We also showcase results when combining our framework with particle MCMC methods. We show that, on six real-world datasets, our approach can learn transition dynamics and latent states more accurately than competing methods.