# Entry Dependent Expert Selection in Distributed Gaussian Processes Using Multilabel Classification

arXiv: https://arxiv.org/abs/2211.09940 · HTML: https://mlsanity.com/paper/2211.09940/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 47 references that are in the map.
- First posted on arXiv 17 Nov 2022.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [A Survey on Distributed Machine Learning](https://mlsanity.com/paper/1912.09789/) (2019)
- [Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning](https://mlsanity.com/paper/2106.02584/) (2021)
- [Distributed Gaussian Processes](https://mlsanity.com/paper/1502.02843/) (2015)
- [Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions](https://mlsanity.com/paper/1410.7827/) (2014)
- [From Distributed Machine Learning to Federated Learning: A Survey](https://mlsanity.com/paper/2104.14362/) (2021)
- [Rates of Convergence for Sparse Variational Gaussian Process Regression](https://mlsanity.com/paper/1903.03571/) (2019)
- [Generalized Robust Bayesian Committee Machine for Large-scale Gaussian Process Regression](https://mlsanity.com/paper/1806.00720/) (2018)
- [Ranking and combining multiple predictors without labeled data](https://mlsanity.com/paper/1303.3257/) (2013)
- [Variational Inference for Gaussian Process Models with Linear Complexity](https://mlsanity.com/paper/1711.10127/) (2017)
- [Unsupervised Ensemble Learning with Dependent Classifiers](https://mlsanity.com/paper/1510.05830/) (2015)
- [Nested Kriging predictions for datasets with large number of observations](https://mlsanity.com/paper/1607.05432/) (2016)
- [ML-Net: multi-label classification of biomedical texts with deep neural networks](https://mlsanity.com/paper/1811.05475/) (2018)
- [Patchwork Kriging for Large-scale Gaussian Process Regression](https://mlsanity.com/paper/1701.06655/) (2017)
- [Deep Learning for Multi-label Classification](https://mlsanity.com/paper/1502.05988/) (2014)
- [An asymptotic analysis of distributed nonparametric methods](https://mlsanity.com/paper/1711.03149/) (2017)
- [Nonparametric and high-dimensional functional graphical models](https://mlsanity.com/paper/2103.10568/) (2021)
- [Bayesian Inference in Nonparanormal Graphical Models](https://mlsanity.com/paper/1806.04334/) (2018)
- [Aggregating Dependent Gaussian Experts in Local Approximation](https://mlsanity.com/paper/2010.08873/) (2020)
- [Gaussian Process Random Fields](https://mlsanity.com/paper/1511.00054/) (2015)
- [Gaussian Experts Selection using Graphical Models](https://mlsanity.com/paper/2102.01496/) (2021)

## Abstract (arXiv)

By distributing the training process, local approximation reduces the cost of the standard Gaussian Process. An ensemble technique combines local predictions from Gaussian experts trained on different partitions of the data. Ensemble methods aggregate models' predictions by assuming a perfect diversity of local predictors. Although it keeps the aggregation tractable, this assumption is often violated in practice. Even though ensemble methods provide consistent results by assuming dependencies between experts, they have a high computational cost, which is cubic in the number of experts involved. By implementing an expert selection strategy, the final aggregation step uses fewer experts and is more efficient. However, a selection approach that assigns a fixed set of experts to each new data point cannot encode the specific properties of each unique data point. This paper proposes a flexible expert selection approach based on the characteristics of entry data points. To this end, we investigate the selection task as a multi-label classification problem where the experts define labels, and each entry point is assigned to some experts. The proposed solution's prediction quality, efficiency, and asymptotic properties are discussed in detail. We demonstrate the efficacy of our method through extensive numerical experiments using synthetic and real-world data sets.