# Distributed Gaussian Processes

arXiv: https://arxiv.org/abs/1502.02843 · HTML: https://mlsanity.com/paper/1502.02843/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Feb 2015.

## Built on by (most cited first, 30 of 84)

- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Model Merging by Uncertainty-Based Gradient Matching](https://mlsanity.com/paper/2310.12808/) (2023)
- [Residual Mixture of Experts](https://mlsanity.com/paper/2204.09636/) (2022)
- [MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance](https://mlsanity.com/paper/2505.03804/) (2025)
- [MoE-Loco: Mixture of Experts for Multitask Locomotion](https://mlsanity.com/paper/2503.08564/) (2025)
- [Local approximate Gaussian process regression for data-driven constitutive laws: Development and comparison with neural networks](https://mlsanity.com/paper/2105.04554/) (2021)
- [Bias-Free Scalable Gaussian Processes via Randomized Truncations](https://mlsanity.com/paper/2102.06695/) (2021)
- [Incremental Ensemble Gaussian Processes](https://mlsanity.com/paper/2110.06777/) (2021)
- [Mixture of Lookup Experts](https://mlsanity.com/paper/2503.15798/) (2025)
- [Training-efficient density quantum machine learning](https://mlsanity.com/paper/2405.20237/) (2024)
- [Stochastic Marginal Likelihood Gradients using Neural Tangent Kernels](https://mlsanity.com/paper/2306.03968/) (2023)
- [MoLAE: Mixture of Latent Experts for Parameter-Efficient Language Models](https://mlsanity.com/paper/2503.23100/) (2025)
- [The Fast Kernel Transform](https://mlsanity.com/paper/2106.04487/) (2021)
- [Leveraging Probabilistic Circuits for Nonparametric Multi-Output Regression](https://mlsanity.com/paper/2106.08687/) (2021)
- [Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling](https://mlsanity.com/paper/2111.14008/) (2021)
- [Scalable Gaussian-process regression and variable selection using Vecchia approximations](https://mlsanity.com/paper/2202.12981/) (2022)
- [Hierarchical Policy Blending as Inference for Reactive Robot Control](https://mlsanity.com/paper/2210.07890/) (2022)
- [Correlated Product of Experts for Sparse Gaussian Process Regression](https://mlsanity.com/paper/2112.09519/) (2021)
- [Neural Operator Variational Inference based on Regularized Stein Discrepancy for Deep Gaussian Processes](https://mlsanity.com/paper/2309.12658/) (2023)
- [Hierarchical Routing Mixture of Experts](https://mlsanity.com/paper/1903.07756/) (2019)
- [Gaussian-Process-based Robot Learning from Demonstration](https://mlsanity.com/paper/2002.09979/) (2020)
- [Recent Advances in Data-Driven Wireless Communication Using Gaussian Processes: A Comprehensive Survey](https://mlsanity.com/paper/2103.10134/) (2021)
- [Federated Bayesian Neural Regression: A Scalable Global Federated Gaussian Process](https://mlsanity.com/paper/2206.06357/) (2022)
- [Sequential Estimation of Gaussian Process-based Deep State-Space Models](https://mlsanity.com/paper/2301.12528/) (2023)
- [Leveraging Locality and Robustness to Achieve Massively Scalable Gaussian Process Regression](https://mlsanity.com/paper/2306.14731/) (2023)
- [No Need to Talk: Asynchronous Mixture of Language Models](https://mlsanity.com/paper/2410.03529/) (2024)
- [Distributed Nonparametric Function Estimation: Optimal Rate of Convergence and Cost of Adaptation](https://mlsanity.com/paper/2107.00179/) (2021)
- [Fully Decentralized, Scalable Gaussian Processes for Multi-Agent Federated Learning](https://mlsanity.com/paper/2203.02865/) (2022)
- [Actually Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2304.05091/) (2023)
- [Cooperative Learning with Gaussian Processes for Euler-Lagrange Systems Tracking Control under Switching Topologies](https://mlsanity.com/paper/2402.03048/) (2024)

## Abstract (arXiv)

To scale Gaussian processes (GPs) to large data sets we introduce the robust Bayesian Committee Machine (rBCM), a practical and scalable product-of-experts model for large-scale distributed GP regression. Unlike state-of-the-art sparse GP approximations, the rBCM is conceptually simple and does not rely on inducing or variational parameters. The key idea is to recursively distribute computations to independent computational units and, subsequently, recombine them to form an overall result. Efficient closed-form inference allows for straightforward parallelisation and distributed computations with a small memory footprint. The rBCM is independent of the computational graph and can be used on heterogeneous computing infrastructures, ranging from laptops to clusters. With sufficient computing resources our distributed GP model can handle arbitrarily large data sets.