# Collaborative and Distributed Bayesian Optimization via Consensus: Showcasing the Power of Collaboration for Optimal Design

arXiv: https://arxiv.org/abs/2306.14348 · HTML: https://mlsanity.com/paper/2306.14348/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 96 references that are in the map.
- First posted on arXiv 25 Jun 2023.

## Builds on (most cited first, 18 of 18)

- [Scalable Bayesian Optimization Using Deep Neural Networks](https://mlsanity.com/paper/1502.05700/) (2015)
- [Federated Bayesian Optimization via Thompson Sampling](https://mlsanity.com/paper/2010.10154/) (2020)
- [The Parallel Knowledge Gradient Method for Batch Bayesian Optimization](https://mlsanity.com/paper/1606.04414/) (2016)
- [Bayesian Optimization for Materials Design with Mixed Quantitative and Qualitative Variables](https://mlsanity.com/paper/1910.01688/) (2019)
- [Uniform Error Bounds for Gaussian Process Regression with Application to Safe Control](https://mlsanity.com/paper/1906.01376/) (2019)
- [Active Learning for Deep Gaussian Process Surrogates](https://mlsanity.com/paper/2012.08015/) (2020)
- [BINOCULARS for Efficient, Nonmyopic Sequential Experimental Design](https://mlsanity.com/paper/1909.04568/) (2019)
- [Differentially Private Federated Bayesian Optimization with Distributed Exploration](https://mlsanity.com/paper/2110.14153/) (2021)
- [Balancing Communication and Computation in Distributed Optimization](https://mlsanity.com/paper/1709.02999/) (2017)
- [Why Non-myopic Bayesian Optimization is Promising and How Far Should We Look-ahead? A Study via Rollout](https://mlsanity.com/paper/1911.01004/) (2019)
- [The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning](https://mlsanity.com/paper/2111.05326/) (2021)
- [Efficient Rollout Strategies for Bayesian Optimization](https://mlsanity.com/paper/2002.10539/) (2020)
- [On Batch Bayesian Optimization](https://mlsanity.com/paper/1911.01032/) (2019)
- [Gaussian Process Optimization with Mutual Information](https://mlsanity.com/paper/1311.4825/) (2013)
- [Active emulation of computer codes with Gaussian processes -- Application to remote sensing](https://mlsanity.com/paper/1912.06552/) (2019)
- [Deep Gaussian Process Emulation using Stochastic Imputation](https://mlsanity.com/paper/2107.01590/) (2021)
- [Bayesian optimization in ab initio nuclear physics](https://mlsanity.com/paper/1902.00941/) (2019)
- [Sequential design of multi-fidelity computer experiments: maximizing the rate of stepwise uncertainty reduction](https://mlsanity.com/paper/2007.13553/) (2020)

## Built on by (most cited first, 9 of 9)

- [LLINBO: Trustworthy LLM-in-the-Loop Bayesian Optimization](https://mlsanity.com/paper/2505.14756/) (2025)
- [Collaborative Bayesian Optimization via Wasserstein Barycenters](https://mlsanity.com/paper/2504.10770/) (2025)
- [Collaborative and Federated Black-box Optimization: A Bayesian Optimization Perspective](https://mlsanity.com/paper/2411.07523/) (2024)
- [Collaborative Contextual Bayesian Optimization](https://mlsanity.com/paper/2604.18912/) (2026)
- [ARCO-BO: Adaptive Resource-aware COllaborative Bayesian Optimization for Heterogeneous Multi-Agent Design](https://mlsanity.com/paper/2510.16652/) (2025)
- [Weighted Wasserstein Barycenter of Gaussian Processes for exotic Bayesian Optimization tasks](https://mlsanity.com/paper/2602.09181/) (2026)
- [ADKO: Agentic Decentralized Knowledge Optimization](https://mlsanity.com/paper/2605.07863/) (2026)
- [Privacy-Aware Collaborative and Distributed Bayesian Optimization](https://mlsanity.com/paper/2607.11600/) (2026)
- [GUIDE-FBO: Guidance via Uncertainty Intervention and Distributional Exchange for Federated Bayesian Optimization](https://mlsanity.com/paper/2609.35038/) (2026)

## Abstract (arXiv)

Optimal design is a critical yet challenging task within many applications. This challenge arises from the need for extensive trial and error, often done through simulations or running field experiments. Fortunately, sequential optimal design, also referred to as Bayesian optimization when using surrogates with a Bayesian flavor, has played a key role in accelerating the design process through efficient sequential sampling strategies. However, a key opportunity exists nowadays. The increased connectivity of edge devices sets forth a new collaborative paradigm for Bayesian optimization. A paradigm whereby different clients collaboratively borrow strength from each other by effectively distributing their experimentation efforts to improve and fast-track their optimal design process. To this end, we bring the notion of consensus to Bayesian optimization, where clients agree (i.e., reach a consensus) on their next-to-sample designs. Our approach provides a generic and flexible framework that can incorporate different collaboration mechanisms. In lieu of this, we propose transitional collaborative mechanisms where clients initially rely more on each other to maneuver through the early stages with scant data, then, at the late stages, focus on their own objectives to get client-specific solutions. Theoretically, we show the sub-linear growth in regret for our proposed framework. Empirically, through simulated datasets and a real-world collaborative sensor design experiment, we show that our framework can effectively accelerate and improve the optimal design process and benefit all participants.