# Combining Pseudo-Point and State Space Approximations for Sum-Separable Gaussian Processes

arXiv: https://arxiv.org/abs/2106.10210 · HTML: https://mlsanity.com/paper/2106.10210/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 55 references that are in the map.
- First posted on arXiv 18 Jun 2021.

## Builds on (most cited first, 20 of 20)

- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Scalable Variational Gaussian Process Classification](https://mlsanity.com/paper/1411.2005/) (2014)
- [Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)](https://mlsanity.com/paper/1503.01057/) (2015)
- [Variational Fourier features for Gaussian processes](https://mlsanity.com/paper/1611.06740/) (2016)
- [Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models](https://mlsanity.com/paper/1703.04265/) (2017)
- [Rates of Convergence for Sparse Variational Gaussian Process Regression](https://mlsanity.com/paper/1903.03571/) (2019)
- [On Sparse variational methods and the Kullback-Leibler divergence between stochastic processes](https://mlsanity.com/paper/1504.07027/) (2015)
- [Don't Unroll Adjoint: Differentiating SSA-Form Programs](https://mlsanity.com/paper/1810.07951/) (2018)
- [Sparse Gaussian Processes with Spherical Harmonic Features](https://mlsanity.com/paper/2006.16649/) (2020)
- [Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Models](https://mlsanity.com/paper/1807.04489/) (2018)
- [Temporal Parallelization of Bayesian Smoothers](https://mlsanity.com/paper/1905.13002/) (2019)
- [Product Kernel Interpolation for Scalable Gaussian Processes](https://mlsanity.com/paper/1802.08903/) (2018)
- [Sparse Gaussian Process Variational Autoencoders](https://mlsanity.com/paper/2010.10177/) (2020)
- [Fast Variational Learning in State-Space Gaussian Process Models](https://mlsanity.com/paper/2007.04731/) (2020)
- [Doubly Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2001.05363/) (2020)
- [Parallelizable sparse inverse formulation Gaussian processes (SpInGP)](https://mlsanity.com/paper/1610.08035/) (2016)
- [State Space Expectation Propagation: Efficient Inference Schemes for Temporal Gaussian Processes](https://mlsanity.com/paper/2007.05994/) (2020)
- [Variational Orthogonal Features](https://mlsanity.com/paper/2006.13170/) (2020)
- [Band-Limited Gaussian Processes: The Sinc Kernel](https://mlsanity.com/paper/1909.07279/) (2019)
- [Infinite-Horizon Gaussian Processes](https://mlsanity.com/paper/1811.06588/) (2018)

## Built on by (most cited first, 7 of 7)

- [Spatio-Temporal Variational Gaussian Processes](https://mlsanity.com/paper/2111.01732/) (2021)
- [Markovian Gaussian Process Variational Autoencoders](https://mlsanity.com/paper/2207.05543/) (2022)
- [Bayes-Newton Methods for Approximate Bayesian Inference with PSD Guarantees](https://mlsanity.com/paper/2111.01721/) (2021)
- [The Rank-Reduced Kalman Filter: Approximate Dynamical-Low-Rank Filtering In High Dimensions](https://mlsanity.com/paper/2306.07774/) (2023)
- [Sequential Monte Carlo Learning for Time Series Structure Discovery](https://mlsanity.com/paper/2307.09607/) (2023)
- [Diffusion-aware Censored Gaussian Processes for Demand Modelling](https://mlsanity.com/paper/2501.12354/) (2025)
- [A Factor Graph Approach to Scalable Multi-Output Gaussian Process Regression](https://mlsanity.com/paper/2608.11917/) (2026)

## Abstract (arXiv)

Gaussian processes (GPs) are important probabilistic tools for inference and learning in spatio-temporal modelling problems such as those in climate science and epidemiology. However, existing GP approximations do not simultaneously support large numbers of off-the-grid spatial data-points and long time-series which is a hallmark of many applications. Pseudo-point approximations, one of the gold-standard methods for scaling GPs to large data sets, are well suited for handling off-the-grid spatial data. However, they cannot handle long temporal observation horizons effectively reverting to cubic computational scaling in the time dimension. State space GP approximations are well suited to handling temporal data, if the temporal GP prior admits a Markov form, leading to linear complexity in the number of temporal observations, but have a cubic spatial cost and cannot handle off-the-grid spatial data. In this work we show that there is a simple and elegant way to combine pseudo-point methods with the state space GP approximation framework to get the best of both worlds. The approach hinges on a surprising conditional independence property which applies to space--time separable GPs. We demonstrate empirically that the combined approach is more scalable and applicable to a greater range of spatio-temporal problems than either method on its own.