# Correlated Product of Experts for Sparse Gaussian Process Regression

arXiv: https://arxiv.org/abs/2112.09519 · HTML: https://mlsanity.com/paper/2112.09519/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 17 Dec 2021.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Gaussian Process Kernels for Pattern Discovery and Extrapolation](https://mlsanity.com/paper/1302.4245/) (2013)
- [Exact Gaussian Processes on a Million Data Points](https://mlsanity.com/paper/1903.08114/) (2019)
- [Distributed Gaussian Processes](https://mlsanity.com/paper/1502.02843/) (2015)
- [Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets](https://mlsanity.com/paper/1406.7343/) (2014)
- [Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions](https://mlsanity.com/paper/1410.7827/) (2014)
- [A general framework for Vecchia approximations of Gaussian processes](https://mlsanity.com/paper/1708.06302/) (2017)
- [Streaming Sparse Gaussian Process Approximations](https://mlsanity.com/paper/1705.07131/) (2017)
- [Generalized Robust Bayesian Committee Machine for Large-scale Gaussian Process Regression](https://mlsanity.com/paper/1806.00720/) (2018)
- [Vecchia approximations of Gaussian-process predictions](https://mlsanity.com/paper/1805.03309/) (2018)
- [Nested Kriging predictions for datasets with large number of observations](https://mlsanity.com/paper/1607.05432/) (2016)
- [Deep Structured Mixtures of Gaussian Processes](https://mlsanity.com/paper/1910.04536/) (2019)
- [Time series forecasting with Gaussian Processes needs priors](https://mlsanity.com/paper/2009.08102/) (2020)
- [Banded Matrix Operators for Gaussian Markov Models in the Automatic Differentiation Era](https://mlsanity.com/paper/1902.10078/) (2019)
- [Recursive Estimation for Sparse Gaussian Process Regression](https://mlsanity.com/paper/1905.11711/) (2019)
- [Parallelizable sparse inverse formulation Gaussian processes (SpInGP)](https://mlsanity.com/paper/1610.08035/) (2016)
- [A Unifying Framework for Gaussian Process Pseudo-Point Approximations using Power Expectation Propagation](https://mlsanity.com/paper/1605.07066/) (2016)

## Built on by (most cited first, 7 of 7)

- [Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processes](https://mlsanity.com/paper/2203.01459/) (2022)
- [Exact Gaussian Processes for Massive Datasets via Non-Stationary Sparsity-Discovering Kernels](https://mlsanity.com/paper/2205.09070/) (2022)
- [Learning Choice Functions with Gaussian Processes](https://mlsanity.com/paper/2302.00406/) (2023)
- [dynoGP: Deep Gaussian Processes for dynamic system identification](https://mlsanity.com/paper/2502.05620/) (2025)
- [Modeling Complex Disease Trajectories using Deep Generative Models with Semi-Supervised Latent Processes](https://mlsanity.com/paper/2311.08149/) (2023)
- [Towards AI-Based Precision Oncology: A Machine Learning Framework for Personalized Counterfactual Treatment Suggestions based on Multi-Omics Data](https://mlsanity.com/paper/2402.12190/) (2024)
- [Semi-Supervised Generative Models for Disease Trajectories: A Case Study on Systemic Sclerosis](https://mlsanity.com/paper/2407.11427/) (2024)

## Abstract (arXiv)

Gaussian processes (GPs) are an important tool in machine learning and statistics with applications ranging from social and natural science through engineering. They constitute a powerful kernelized non-parametric method with well-calibrated uncertainty estimates, however, off-the-shelf GP inference procedures are limited to datasets with several thousand data points because of their cubic computational complexity. For this reason, many sparse GPs techniques have been developed over the past years. In this paper, we focus on GP regression tasks and propose a new approach based on aggregating predictions from several local and correlated experts. Thereby, the degree of correlation between the experts can vary between independent up to fully correlated experts. The individual predictions of the experts are aggregated taking into account their correlation resulting in consistent uncertainty estimates. Our method recovers independent Product of Experts, sparse GP and full GP in the limiting cases. The presented framework can deal with a general kernel function and multiple variables, and has a time and space complexity which is linear in the number of experts and data samples, which makes our approach highly scalable. We demonstrate superior performance, in a time vs. accuracy sense, of our proposed method against state-of-the-art GP approximation methods for synthetic as well as several real-world datasets with deterministic and stochastic optimization.