# Embarrassingly Parallel Inference for Gaussian Processes

arXiv: https://arxiv.org/abs/1702.08420 · HTML: https://mlsanity.com/paper/1702.08420/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts.
- First posted on arXiv 27 Feb 2017.

## Placed in

- Mixture-of-experts / [MoE theory and gating functions](https://mlsanity.com/moe/moe-theory-and-gating-functions/)

## Built on by (most cited first, 6 of 6)

- [Leveraging Probabilistic Circuits for Nonparametric Multi-Output Regression](https://mlsanity.com/paper/2106.08687/) (2021)
- [Mixtures of Gaussian Process Experts with SMC2](https://mlsanity.com/paper/2208.12830/) (2022)
- [Healing Products of Gaussian Processes](https://mlsanity.com/paper/2102.07106/) (2021)
- [A Fast Kernel-based Conditional Independence test with Application to Causal Discovery](https://mlsanity.com/paper/2505.11085/) (2025)
- [Temporal Gaussian Process Regression in Logarithmic Time](https://mlsanity.com/paper/2102.09964/) (2021)
- [Online Student-t Processes with an Overall-local Scale Structure for Modelling Non-stationary Data](https://mlsanity.com/paper/2311.00564/) (2023)

## Abstract (arXiv)

Training Gaussian process-based models typically involves an  O(N3) computational bottleneck due to inverting the covariance matrix. Popular methods for overcoming this matrix inversion problem cannot adequately model all types of latent functions, and are often not parallelizable. However, judicious choice of model structure can ameliorate this problem. A mixture-of-experts model that uses a mixture of K Gaussian processes offers modeling flexibility and opportunities for scalable inference. Our embarrassingly parallel algorithm combines low-dimensional matrix inversions with importance sampling to yield a flexible, scalable mixture-of-experts model that offers comparable performance to Gaussian process regression at a much lower computational cost.