mlsanityData to 14 Oct 2026

Federated Bayesian Optimization via Thompson Sampling

arXiv 2010.10154 ↗First posted 20 Oct 2020cs.LG cs.AIcited by 77

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization (BO) is a prominent approach to optimizing expensive-to-evaluate black-box functions. The massive computational capability of edge devices such as mobile phones, coupled with privacy concerns, has led to a surging interest in federated learning (FL) which focuses on collaborative training of deep neural networks (DNNs) via first-order optimization techniques. However, some common machine learning tasks such as hyperparameter tuning of DNNs lack access to gradients and thus require zeroth-order/black-box optimization. This hints at the possibility of extending BO to the FL setting (FBO) for agents to collaborate in these black-box optimization tasks. This paper presents federated Thompson sampling (FTS) which overcomes a number of key challenges of FBO and FL in a principled way: We (a) use random Fourier features to approximate the Gaussian process surrogate model used in BO, which naturally produces the parameters to be exchanged between agents, (b) design FTS based on Thompson sampling, which significantly reduces the number of parameters to be exchanged, and (c) provide a theoretical convergence guarantee that is robust against heterogeneous agents, which is a major challenge in FL and FBO. We empirically demonstrate the effectiveness of FTS in terms of communication efficiency, computational efficiency, and practical performance.

Key facts. Cited by 77 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 13; 2022: 22; 2023: 19; 2024: 12; 2025: 7; 2026: 3.

77 papers.

Sort:

By area:

  1. 2021Model-Contrastive Federated Learningcited by 766
  2. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  3. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 149
  4. 2021Personalized Federated Learning with Gaussian Processescited by 76
  5. 2022Recent Advances in Bayesian Optimizationcited by 76
  6. 2022Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learningcited by 59
  7. 2021Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharingcited by 58
  8. 2021Fault-Tolerant Federated Reinforcement Learning with Theoretical Guaranteecited by 54
  9. 2022FederatedScope: A Flexible Federated Learning Platform for Heterogeneitycited by 53
  10. 2021Differentially Private Federated Bayesian Optimization with Distributed Explorationcited by 32
  11. 2022Sample-Then-Optimize Batch Neural Thompson Samplingcited by 25
  12. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23
38 more
  1. 2023Bayesian Federated Learning: A Surveycited by 22
  2. 2021GIFAIR-FL: A Framework for Group and Individual Fairness in Federated Learningcited by 21
  3. 2022Unifying and Boosting Gradient-Based Training-Free Neural Architecture Searchcited by 17
  4. 2022Federated Neural Banditscited by 17
  5. 2020Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimizationcited by 16
  6. 2021FLoRA: Single-shot Hyper-parameter Optimization for Federated Learningcited by 14
  7. 2023Exploiting Label Skews in Federated Learning with Model Concatenationcited by 14
  8. 2025LLINBO: Trustworthy LLM-in-the-Loop Bayesian Optimizationcited by 14
  9. 2023Quantum Bayesian Optimizationcited by 12
  10. 2021Unsupervised Reservoir Computing for Solving Ordinary Differential Equationscited by 11
  11. 2021A Payload Optimization Method for Federated Recommender Systemscited by 10
  12. 2022Bayesian Optimization under Stochastic Delayed Feedbackcited by 10
  13. 2023Collaborative and Distributed Bayesian Optimization via Consensus: Showcasing the Power of Collaboration for Optimal Designcited by 9
  14. 2021Evaluation of Hyperparameter-Optimization Approaches in an Industrial Federated Learning Systemcited by 8
  15. 2022On Provably Robust Meta-Bayesian Optimizationcited by 8
  16. 2022Differentially Private Linear Bandits with Partial Distributed Feedbackcited by 8
  17. 2023Batch Bayesian Optimization for Replicable Experimental Designcited by 8
  18. 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applicationscited by 8
  19. 2022On Noisy Evaluation in Federated Hyperparameter Tuningcited by 7
  20. 2025Collaborative Bayesian Optimization via Wasserstein Barycenterscited by 7
  21. 2021Optimum-statistical Collaboration Towards General and Efficient Black-box Optimizationcited by 6
  22. 2022FedHPO-B: A Benchmark Suite for Federated Hyperparameter Optimizationcited by 6
  23. 2022An Empirical Study of Personalized Federated Learningcited by 6
  24. 2022Collaborative Learning in Kernel-based Bandits for Distributed Userscited by 6
  25. 2022Federated X-Armed Banditcited by 5
  26. 2022Federated Bayesian Neural Regression: A Scalable Global Federated Gaussian Processcited by 5
  27. 2023Training-Free Neural Active Learning with Initialization-Robustness Guaranteescited by 5
  28. 2024Collaborative and Federated Black-box Optimization: A Bayesian Optimization Perspectivecited by 5
  29. 2022A Secure Federated Data-Driven Evolutionary Multi-objective Optimization Algorithmcited by 4
  30. 2023Optimizing Algorithms From Pairwise User Preferencescited by 4
  31. 2024Rendering Wireless Environments Useful for Gradient Estimators: A Zero-Order Stochastic Federated Learning Methodcited by 4
  32. 2024TS-RSR: A provably efficient approach for batch Bayesian Optimizationcited by 4
  33. 2024Robustifying and Boosting Training-Free Neural Architecture Searchcited by 4
  34. 2024FedSI: Federated Subnetwork Inference for Efficient Uncertainty Quantificationcited by 4
  35. 2022Single-shot Hyper-parameter Optimization for Federated Learning: A General Algorithm & Analysiscited by 3
  36. 2022RcTorch: a PyTorch Reservoir Computing Package with Automated Hyper-Parameter Optimizationcited by 3
  37. 2023(Private) Kernelized Bandits with Distributed Biased Feedbackcited by 3
  38. 2024Adaptive Active Inference Agents for Heterogeneous and Lifelong Federated Learningcited by 3

50 most cited of 77 All 77 (JSON)