# Mixtures of Gaussian Process Experts with SMC2

arXiv: https://arxiv.org/abs/2208.12830 · HTML: https://mlsanity.com/paper/2208.12830/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 77 references that are in the map.
- First posted on arXiv 26 Aug 2022.

## Builds on (most cited first, 12 of 12)

- [Distributed Gaussian Processes](https://mlsanity.com/paper/1502.02843/) (2015)
- [Generalized Robust Bayesian Committee Machine for Large-scale Gaussian Process Regression](https://mlsanity.com/paper/1806.00720/) (2018)
- [An invitation to sequential Monte Carlo samplers](https://mlsanity.com/paper/2007.11936/) (2020)
- [Non-Stationary Gaussian Process Regression with Hamiltonian Monte Carlo](https://mlsanity.com/paper/1508.04319/) (2015)
- [Gaussian process regression for forecasting battery state of health](https://mlsanity.com/paper/1703.05687/) (2017)
- [The Dependent Dirichlet Process and Related Models](https://mlsanity.com/paper/2007.06129/) (2020)
- [Marginalizing Gaussian Process Hyperparameters using Sequential Monte Carlo](https://mlsanity.com/paper/1502.01908/) (2015)
- [Embarrassingly Parallel Inference for Gaussian Processes](https://mlsanity.com/paper/1702.08420/) (2017)
- [Dropout Regularization in Hierarchical Mixture of Experts](https://mlsanity.com/paper/1812.10158/) (2018)
- [Sparse online variational Bayesian regression](https://mlsanity.com/paper/2102.12261/) (2021)
- [Non-stationary Gaussian process discriminant analysis with variable selection for high-dimensional functional data](https://mlsanity.com/paper/2109.14171/) (2021)
- [Jeffreys priors for mixture estimation: properties and alternatives](https://mlsanity.com/paper/1706.02563/) (2017)

## Built on by (most cited first, 3 of 3)

- [Log-Gaussian Gamma Processes for Training Bayesian Neural Networks in Raman and CARS Spectroscopies](https://mlsanity.com/paper/2310.08055/) (2023)
- [Online Student-t Processes with an Overall-local Scale Structure for Modelling Non-stationary Data](https://mlsanity.com/paper/2311.00564/) (2023)
- [Covariance-Boosted Gaussian Processes for Spatiotemporal Irregularities](https://mlsanity.com/paper/2607.23018/) (2026)

## Abstract (arXiv)

Gaussian processes are a key component of many flexible statistical and machine learning models. However, they exhibit cubic computational complexity and high memory constraints due to the need of inverting and storing a full covariance matrix. To circumvent this, mixtures of Gaussian process experts have been considered where data points are assigned to independent experts, reducing the complexity by allowing inference based on smaller, local covariance matrices. Moreover, mixtures of Gaussian process experts substantially enrich the model's flexibility, allowing for behaviors such as non-stationarity, heteroscedasticity, and discontinuities. In this work, we construct a novel inference approach based on nested sequential Monte Carlo samplers to simultaneously infer both the gating network and Gaussian process expert parameters. This greatly improves inference compared to importance sampling, particularly in settings when a stationary Gaussian process is inappropriate, while still being thoroughly parallelizable.