# Scalable Multi-Output Gaussian Processes with Stochastic Variational Inference

arXiv: https://arxiv.org/abs/2407.02476 · HTML: https://mlsanity.com/paper/2407.02476/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 59 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Builds on (most cited first, 25 of 25)

- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series](https://mlsanity.com/paper/1905.12374/) (2019)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Doubly Stochastic Variational Inference for Deep Gaussian Processes](https://mlsanity.com/paper/1705.08933/) (2017)
- [Gaussian Process Kernels for Pattern Discovery and Extrapolation](https://mlsanity.com/paper/1302.4245/) (2013)
- [Spatio-Temporal Variational Gaussian Processes](https://mlsanity.com/paper/2111.01732/) (2021)
- [Spectral Mixture Kernels for Multi-Output Gaussian Processes](https://mlsanity.com/paper/1709.01298/) (2017)
- [Heterogeneous Multi-output Gaussian Process Prediction](https://mlsanity.com/paper/1805.07633/) (2018)
- [Scalable Exact Inference in Multi-Output Gaussian Processes](https://mlsanity.com/paper/1911.06287/) (2019)
- [Multi-task Learning for Aggregated Data using Gaussian Processes](https://mlsanity.com/paper/1906.09412/) (2019)
- [Orthogonally Decoupled Variational Gaussian Processes](https://mlsanity.com/paper/1809.08820/) (2018)
- [Sparse within Sparse Gaussian Processes using Neighbor Information](https://mlsanity.com/paper/2011.05041/) (2020)
- [Efficient Modeling of Latent Information in Supervised Learning using Gaussian Processes](https://mlsanity.com/paper/1705.09862/) (2017)
- [Doubly Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2001.05363/) (2020)
- [The Gaussian Process Autoregressive Regression Model (GPAR)](https://mlsanity.com/paper/1802.07182/) (2018)
- [Nonstationary multi-output Gaussian processes via harmonizable spectral mixtures](https://mlsanity.com/paper/2202.09233/) (2022)
- [Efficient Inference in Multi-task Cox Process Models](https://mlsanity.com/paper/1805.09781/) (2018)
- [Scalable Multi-Task Gaussian Processes with Neural Embedding of Coregionalization](https://mlsanity.com/paper/2109.09261/) (2021)
- [Generalised Gaussian Process Latent Variable Models (GPLVM) with Stochastic Variational Inference](https://mlsanity.com/paper/2202.12979/) (2022)
- [A Fully Natural Gradient Scheme for Improving Inference of the Heterogeneous Multi-Output Gaussian Process Model](https://mlsanity.com/paper/1911.10225/) (2019)
- [A conditional one-output likelihood formulation for multitask Gaussian processes](https://mlsanity.com/paper/2006.03495/) (2020)
- [Latent Variable Multi-output Gaussian Processes for Hierarchical Datasets](https://mlsanity.com/paper/2308.16822/) (2023)
- [Adaptive Cholesky Gaussian Processes](https://mlsanity.com/paper/2202.10769/) (2022)

## Built on by (most cited first, 2 of 2)

- [Gaussian Process Latent Factor Regression for Low-Data, High-Dimensional Output Problems](https://mlsanity.com/paper/2606.06576/) (2026)
- [Transformed Latent Variable Multi-Output Gaussian Processes](https://mlsanity.com/paper/2605.05133/) (2026)

## Abstract (arXiv)

The Multi-Output Gaussian Process is is a popular tool for modelling data from multiple sources. A typical choice to build a covariance function for a MOGP is the Linear Model of Coregionalization (LMC) which parametrically models the covariance between outputs. The Latent Variable MOGP (LV-MOGP) generalises this idea by modelling the covariance between outputs using a kernel applied to latent variables, one per output, leading to a flexible MOGP model that allows efficient generalization to new outputs with few data points. Computational complexity in LV-MOGP grows linearly with the number of outputs, which makes it unsuitable for problems with a large number of outputs. In this paper, we propose a stochastic variational inference approach for the LV-MOGP that allows mini-batches for both inputs and outputs, making computational complexity per training iteration independent of the number of outputs.