# Hierarchical Mixture-of-Experts Model for Large-Scale Gaussian Process Regression

arXiv: https://arxiv.org/abs/1412.3078 · HTML: https://mlsanity.com/paper/1412.3078/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2014.

## Built on by (most cited first, 15 of 15)

- [Redesigning Multi-Scale Neural Network for Crowd Counting](https://mlsanity.com/paper/2208.02894/) (2022)
- [Breaking the gridlock in Mixture-of-Experts: Consistent and Efficient Algorithms](https://mlsanity.com/paper/1802.07417/) (2018)
- [Ensemble and Mixture-of-Experts DeepONets For Operator Learning](https://mlsanity.com/paper/2405.11907/) (2024)
- [Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling](https://mlsanity.com/paper/2111.14008/) (2021)
- [Recent Advances in Data-Driven Wireless Communication Using Gaussian Processes: A Comprehensive Survey](https://mlsanity.com/paper/2103.10134/) (2021)
- [Fully Decentralized, Scalable Gaussian Processes for Multi-Agent Federated Learning](https://mlsanity.com/paper/2203.02865/) (2022)
- [Gaussian Process-Gated Hierarchical Mixtures of Experts](https://mlsanity.com/paper/2302.04947/) (2023)
- [Adaptive Resolution for Finite-Rank Gaussian Processes](https://mlsanity.com/paper/2505.24066/) (2025)
- [Fast Gaussian Process Approximations for Autocorrelated Data](https://mlsanity.com/paper/2512.02925/) (2025)
- [Modular Gaussian Processes for Transfer Learning](https://mlsanity.com/paper/2110.13515/) (2021)
- [Quality or Quantity? Error-Informed Selective Online Learning with Gaussian Processes in Multi-Agent Systems: Extended Version](https://mlsanity.com/paper/2601.14275/) (2026)
- [Empirical Asset Pricing via Ensemble Gaussian Process Regression](https://mlsanity.com/paper/2212.01048/) (2022)
- [Aggregation Models with Optimal Weights for Distributed Gaussian Processes](https://mlsanity.com/paper/2408.00955/) (2024)
- [A Partitioned Sparse Variational Gaussian Process for Fast, Distributed Spatial Modeling](https://mlsanity.com/paper/2507.16771/) (2025)
- [Personalized Federated Hierarchical Gaussian Processes for Privacy-Preserving Modeling of Heterogeneous Distributed Systems](https://mlsanity.com/paper/2609.19337/) (2026)

## Abstract (arXiv)

We propose a practical and scalable Gaussian process model for large-scale nonlinear probabilistic regression. Our mixture-of-experts model is conceptually simple and hierarchically recombines computations for an overall approximation of a full Gaussian process. Closed-form and distributed computations allow for efficient and massive parallelisation while keeping the memory consumption small. Given sufficient computing resources, our model can handle arbitrarily large data sets, without explicit sparse approximations. We provide strong experimental evidence that our model can be applied to large data sets of sizes far beyond millions. Hence, our model has the potential to lay the foundation for general large-scale Gaussian process research.