# Spatio-Temporal Variational Gaussian Processes

arXiv: https://arxiv.org/abs/2111.01732 · HTML: https://mlsanity.com/paper/2111.01732/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 55 references that are in the map.
- First posted on arXiv 2 Nov 2021.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Scalable Variational Gaussian Process Classification](https://mlsanity.com/paper/1411.2005/) (2014)
- [Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)](https://mlsanity.com/paper/1503.01057/) (2015)
- [GPflow: A Gaussian process library using TensorFlow](https://mlsanity.com/paper/1610.08733/) (2016)
- [Exact Gaussian Processes on a Million Data Points](https://mlsanity.com/paper/1903.08114/) (2019)
- [Variational Fourier features for Gaussian processes](https://mlsanity.com/paper/1611.06740/) (2016)
- [Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets](https://mlsanity.com/paper/1406.7343/) (2014)
- [Understanding Probabilistic Sparse Gaussian Process Approximations](https://mlsanity.com/paper/1606.04820/) (2016)
- [Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models](https://mlsanity.com/paper/1703.04265/) (2017)
- [Natural Gradients in Practice: Non-Conjugate Variational Inference in Gaussian Process Models](https://mlsanity.com/paper/1803.09151/) (2018)
- [Temporal Parallelization of Bayesian Smoothers](https://mlsanity.com/paper/1905.13002/) (2019)
- [Scalable Exact Inference in Multi-Output Gaussian Processes](https://mlsanity.com/paper/1911.06287/) (2019)
- [Fast Variational Learning in State-Space Gaussian Process Models](https://mlsanity.com/paper/2007.04731/) (2020)
- [Doubly Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2001.05363/) (2020)
- [State Space Gaussian Processes with Non-Gaussian Likelihood](https://mlsanity.com/paper/1802.04846/) (2018)
- [Banded Matrix Operators for Gaussian Markov Models in the Automatic Differentiation Era](https://mlsanity.com/paper/1902.10078/) (2019)
- [State Space Expectation Propagation: Efficient Inference Schemes for Temporal Gaussian Processes](https://mlsanity.com/paper/2007.05994/) (2020)
- [Sparse Algorithms for Markovian Gaussian Processes](https://mlsanity.com/paper/2103.10710/) (2021)
- [Combining Pseudo-Point and State Space Approximations for Sum-Separable Gaussian Processes](https://mlsanity.com/paper/2106.10210/) (2021)
- [AutoGP: Exploring the Capabilities and Limitations of Gaussian Process Models](https://mlsanity.com/paper/1610.05392/) (2016)
- [Efficient Inference in Multi-task Cox Process Models](https://mlsanity.com/paper/1805.09781/) (2018)
- [Scalable high-resolution forecasting of sparse spatiotemporal events with kernel methods: a winning solution to the NIJ "Real-Time Crime Forecasting Challenge"](https://mlsanity.com/paper/1801.02858/) (2018)
- [End-to-End Probabilistic Inference for Nonstationary Audio Analysis](https://mlsanity.com/paper/1901.11436/) (2019)

## Built on by (most cited first, 30 of 32)

- [Markovian Gaussian Process Variational Autoencoders](https://mlsanity.com/paper/2207.05543/) (2022)
- [Computation-Aware Kalman Filtering and Smoothing](https://mlsanity.com/paper/2405.08971/) (2024)
- [Scalable Spatiotemporal Prediction with Bayesian Neural Fields](https://mlsanity.com/paper/2403.07657/) (2024)
- [Revisiting Kernel Attention with Correlated Gaussian Process Representation](https://mlsanity.com/paper/2502.20525/) (2025)
- [Robust and Conjugate Spatio-Temporal Gaussian Processes](https://mlsanity.com/paper/2502.02450/) (2025)
- [eXponential FAmily Dynamical Systems (XFADS): Large-scale nonlinear Gaussian state-space modeling](https://mlsanity.com/paper/2403.01371/) (2024)
- [Physics-Informed Variational State-Space Gaussian Processes](https://mlsanity.com/paper/2409.13876/) (2024)
- [Linear Time GPs for Inferring Latent Trajectories from Neural Spike Trains](https://mlsanity.com/paper/2306.01802/) (2023)
- [The Rank-Reduced Kalman Filter: Approximate Dynamical-Low-Rank Filtering In High Dimensions](https://mlsanity.com/paper/2306.07774/) (2023)
- [Sequential Monte Carlo Learning for Time Series Structure Discovery](https://mlsanity.com/paper/2307.09607/) (2023)
- [SING: SDE Inference via Natural Gradients](https://mlsanity.com/paper/2506.17796/) (2025)
- [Actually Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2304.05091/) (2023)
- [Gaussian Processes for Monitoring Air-Quality in Kampala](https://mlsanity.com/paper/2311.16625/) (2023)
- [Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modelling](https://mlsanity.com/paper/2505.16481/) (2025)
- [Scalable Multi-Output Gaussian Processes with Stochastic Variational Inference](https://mlsanity.com/paper/2407.02476/) (2024)
- [Short-term Prediction and Filtering of Solar Power Using State-Space Gaussian Processes](https://mlsanity.com/paper/2302.00388/) (2023)
- [Efficient Sensor Placement from Regression with Sparse Gaussian Processes in Continuous and Discrete Spaces](https://mlsanity.com/paper/2303.00028/) (2023)
- [STACI: Spatio-Temporal Aleatoric Conformal Inference](https://mlsanity.com/paper/2505.21658/) (2025)
- [BALLAST: Bayesian Active Learning with Look-ahead Amendment for Sea-drifter Trajectories under Spatio-Temporal Vector Fields](https://mlsanity.com/paper/2509.26005/) (2025)
- [UrbanFM: Scaling Urban Spatio-Temporal Foundation Models](https://mlsanity.com/paper/2602.20677/) (2026)
- [Spatio-Temporal Gaussian Process for Building Terrain-Incorporating Wind Power Curves](https://mlsanity.com/paper/2607.00051/) (2026)
- [Deep Functional Factor Models: Forecasting High-Dimensional Functional Time Series via Bayesian Nonparametric Factorization](https://mlsanity.com/paper/2305.14543/) (2023)
- [Deep Gaussian Markov Random Fields for Graph-Structured Dynamical Systems](https://mlsanity.com/paper/2306.08445/) (2023)
- [Learning Coupled Subspaces for Multi-Condition Spike Data](https://mlsanity.com/paper/2410.19153/) (2024)
- [Diffusion-aware Censored Gaussian Processes for Demand Modelling](https://mlsanity.com/paper/2501.12354/) (2025)
- [Effect Decomposition of Functional-Output Computer Experiments via Orthogonal Additive Gaussian Processes](https://mlsanity.com/paper/2506.12701/) (2025)
- [Spatiodynamic inference using vision-based generative modelling](https://mlsanity.com/paper/2507.22256/) (2025)
- [A New Lens on Homelessness: Daily Tent Monitoring with 311 Calls and Street Images](https://mlsanity.com/paper/2508.06409/) (2025)
- [Tackling Incomplete Data in Air Quality Prediction: A Bayesian Deep Learning Framework for Uncertainty Quantification](https://mlsanity.com/paper/2511.02175/) (2025)
- [Designing an Optimal Sensor Network via Minimizing Information Loss](https://mlsanity.com/paper/2512.05940/) (2025)

## Abstract (arXiv)

We introduce a scalable approach to Gaussian process inference that combines spatio-temporal filtering with natural gradient variational inference, resulting in a non-conjugate GP method for multivariate data that scales linearly with respect to time. Our natural gradient approach enables application of parallel filtering and smoothing, further reducing the temporal span complexity to be logarithmic in the number of time steps. We derive a sparse approximation that constructs a state-space model over a reduced set of spatial inducing points, and show that for separable Markov kernels the full and sparse cases exactly recover the standard variational GP, whilst exhibiting favourable computational properties. To further improve the spatial scaling we propose a mean-field assumption of independence between spatial locations which, when coupled with sparsity and parallelisation, leads to an efficient and accurate method for large spatio-temporal problems.