# Scalable Exact Inference in Multi-Output Gaussian Processes

arXiv: https://arxiv.org/abs/1911.06287 · HTML: https://mlsanity.com/paper/1911.06287/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2019.

## Built on by (most cited first, 25 of 25)

- [Bayesian Optimization with High-Dimensional Outputs](https://mlsanity.com/paper/2106.12997/) (2021)
- [Spatio-Temporal Variational Gaussian Processes](https://mlsanity.com/paper/2111.01732/) (2021)
- [Practical Conditional Neural Processes Via Tractable Dependent Predictions](https://mlsanity.com/paper/2203.08775/) (2022)
- [Deep Learning for Bayesian Optimization of Scientific Problems with High-Dimensional Structure](https://mlsanity.com/paper/2104.11667/) (2021)
- [Leveraging Probabilistic Circuits for Nonparametric Multi-Output Regression](https://mlsanity.com/paper/2106.08687/) (2021)
- [Efficient Transformed Gaussian Processes for Non-Stationary Dependent Multi-class Classification](https://mlsanity.com/paper/2205.15008/) (2022)
- [Scalable Multi-Task Gaussian Processes with Neural Embedding of Coregionalization](https://mlsanity.com/paper/2109.09261/) (2021)
- [Online Time Series Anomaly Detection with State Space Gaussian Processes](https://mlsanity.com/paper/2201.06763/) (2022)
- [Sequential Bayesian Optimization for Adaptive Informative Path Planning with Multimodal Sensing](https://mlsanity.com/paper/2209.07660/) (2022)
- [Multi-Output Gaussian Process-Based Data Augmentation for Multi-Building and Multi-Floor Indoor Localization](https://mlsanity.com/paper/2202.01980/) (2022)
- [Federated Automatic Latent Variable Selection in Multi-output Gaussian Processes](https://mlsanity.com/paper/2407.16935/) (2024)
- [Regularized Multi-output Gaussian Convolution Process with Domain Adaptation](https://mlsanity.com/paper/2409.02778/) (2024)
- [Modeling massive highly-multivariate nonstationary spatial data with the basis graphical lasso](https://mlsanity.com/paper/2101.02404/) (2021)
- [Scalable Multi-Output Gaussian Processes with Stochastic Variational Inference](https://mlsanity.com/paper/2407.02476/) (2024)
- [Bayesian Inference in High-Dimensional Time-Serieswith the Orthogonal Stochastic Linear Mixing Model](https://mlsanity.com/paper/2106.13379/) (2021)
- [Distributional Gaussian Processes Layers for Out-of-Distribution Detection](https://mlsanity.com/paper/2206.13346/) (2022)
- [Gaussian Process Latent Factor Regression for Low-Data, High-Dimensional Output Problems](https://mlsanity.com/paper/2606.06576/) (2026)
- [RECOWNs: Probabilistic Circuits for Trustworthy Time Series Forecasting](https://mlsanity.com/paper/2106.04148/) (2021)
- [Exact and general decoupled solutions of the LMC Multitask Gaussian Process model](https://mlsanity.com/paper/2310.12032/) (2023)
- [Open-Source High-Speed Flight Surrogate Modeling Framework](https://mlsanity.com/paper/2411.03598/) (2024)
- [Combinatorial Bandit Bayesian Optimization for Tensor Outputs](https://mlsanity.com/paper/2602.00640/) (2026)
- [Transformed Latent Variable Multi-Output Gaussian Processes](https://mlsanity.com/paper/2605.05133/) (2026)
- [A Factor Graph Approach to Scalable Multi-Output Gaussian Process Regression](https://mlsanity.com/paper/2608.11917/) (2026)
- [Personalized Federated Hierarchical Gaussian Processes for Privacy-Preserving Modeling of Heterogeneous Distributed Systems](https://mlsanity.com/paper/2609.19337/) (2026)
- [A Flow Matching Framework for Neural Representational Dissimilarity](https://mlsanity.com/paper/2609.31544/) (2026)

## Abstract (arXiv)

Multi-output Gaussian processes (MOGPs) leverage the flexibility and interpretability of GPs while capturing structure across outputs, which is desirable, for example, in spatio-temporal modelling. The key problem with MOGPs is their computational scaling O(n3 p3), which is cubic in the number of both inputs n (e.g., time points or locations) and outputs p. For this reason, a popular class of MOGPs assumes that the data live around a low-dimensional linear subspace, reducing the complexity to O(n3 m3). However, this cost is still cubic in the dimensionality of the subspace m, which is still prohibitively expensive for many applications. We propose the use of a sufficient statistic of the data to accelerate inference and learning in MOGPs with orthogonal bases. The method achieves linear scaling in m in practice, allowing these models to scale to large m without sacrificing significant expressivity or requiring approximation. This advance opens up a wide range of real-world tasks and can be combined with existing GP approximations in a plug-and-play way. We demonstrate the efficacy of the method on various synthetic and real-world data sets.