# Fully Decentralized, Scalable Gaussian Processes for Multi-Agent Federated Learning

arXiv: https://arxiv.org/abs/2203.02865 · HTML: https://mlsanity.com/paper/2203.02865/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 72 references that are in the map.
- First posted on arXiv 6 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Distributed Gaussian Processes](https://mlsanity.com/paper/1502.02843/) (2015)
- [Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets](https://mlsanity.com/paper/1406.7343/) (2014)
- [Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions](https://mlsanity.com/paper/1410.7827/) (2014)
- [Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems](https://mlsanity.com/paper/1410.1390/) (2014)
- [On the Linear Convergence of the ADMM in Decentralized Consensus Optimization](https://mlsanity.com/paper/1307.5561/) (2013)
- [Generalized Robust Bayesian Committee Machine for Large-scale Gaussian Process Regression](https://mlsanity.com/paper/1806.00720/) (2018)
- [GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning](https://mlsanity.com/paper/1909.00047/) (2019)
- [Multi-Agent Distributed Optimization via Inexact Consensus ADMM](https://mlsanity.com/paper/1402.6065/) (2014)
- [Nested Kriging predictions for datasets with large number of observations](https://mlsanity.com/paper/1607.05432/) (2016)
- [Efficient Informative Sensing using Multiple Robots](https://mlsanity.com/paper/1401.3462/) (2014)
- [Convergence Rate of Distributed ADMM over Networks](https://mlsanity.com/paper/1601.00194/) (2016)
- [Hierarchical Mixture-of-Experts Model for Large-Scale Gaussian Process Regression](https://mlsanity.com/paper/1412.3078/) (2014)
- [Distributed Mapping with Privacy and Communication Constraints: Lightweight Algorithms and Object-based Models](https://mlsanity.com/paper/1702.03435/) (2017)
- [Gaussian Processes for Nonlinear Signal Processing](https://mlsanity.com/paper/1303.2823/) (2013)
- [Wireless Traffic Prediction with Scalable Gaussian Process: Framework, Algorithms, and Verification](https://mlsanity.com/paper/1902.04763/) (2019)
- [Numerical issues in maximum likelihood parameter estimation for Gaussian process interpolation](https://mlsanity.com/paper/2101.09747/) (2021)
- [A Survey of Distributed Optimization Methods for Multi-Robot Systems](https://mlsanity.com/paper/2103.12840/) (2021)
- [Learning of Gaussian Processes in Distributed and Communication Limited Systems](https://mlsanity.com/paper/1705.02627/) (2017)
- [How priors of initial hyperparameters affect Gaussian process regression models](https://mlsanity.com/paper/1605.07906/) (2016)
- [Learning a Spatial Field in Minimum Time with a Team of Robots](https://mlsanity.com/paper/1909.01895/) (2019)

## Built on by (most cited first, 4 of 4)

- [Advancements in Federated Learning: Models, Methods, and Privacy](https://mlsanity.com/paper/2302.11466/) (2023)
- [Practical Privacy-Preserving Gaussian Process Regression via Secret Sharing](https://mlsanity.com/paper/2306.14498/) (2023)
- [A Spatially Informed Gaussian Process UCB Method for Decentralized Coverage Control](https://mlsanity.com/paper/2511.02398/) (2025)
- [Personalized Federated Hierarchical Gaussian Processes for Privacy-Preserving Modeling of Heterogeneous Distributed Systems](https://mlsanity.com/paper/2609.19337/) (2026)

## Abstract (arXiv)

In this paper, we propose decentralized and scalable algorithms for Gaussian process (GP) training and prediction in multi-agent systems. To decentralize the implementation of GP training optimization algorithms, we employ the alternating direction method of multipliers (ADMM). A closed-form solution of the decentralized proximal ADMM is provided for the case of GP hyper-parameter training with maximum likelihood estimation. Multiple aggregation techniques for GP prediction are decentralized with the use of iterative and consensus methods. In addition, we propose a covariance-based nearest neighbor selection strategy that enables a subset of agents to perform predictions. The efficacy of the proposed methods is illustrated with numerical experiments on synthetic and real data.