# Evaluation of Hyperparameter-Optimization Approaches in an Industrial Federated Learning System

arXiv: https://arxiv.org/abs/2110.08202 · HTML: https://mlsanity.com/paper/2110.08202/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 15 references that are in the map.
- First posted on arXiv 15 Oct 2021.

## Builds on (most cited first, 6 of 6)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [The Non-IID Data Quagmire of Decentralized Machine Learning](https://mlsanity.com/paper/1910.00189/) (2019)
- [Federated Bayesian Optimization via Thompson Sampling](https://mlsanity.com/paper/2010.10154/) (2020)

## Built on by (most cited first, 8 of 8)

- [Differentially Private Federated Bayesian Optimization with Distributed Exploration](https://mlsanity.com/paper/2110.14153/) (2021)
- [Federated Neural Bandits](https://mlsanity.com/paper/2205.14309/) (2022)
- [On Noisy Evaluation in Federated Hyperparameter Tuning](https://mlsanity.com/paper/2212.08930/) (2022)
- [Federated Learning for Predictive Maintenance and Quality Inspection in Industrial Applications](https://mlsanity.com/paper/2304.11101/) (2023)
- [Federated Hypergradient Descent](https://mlsanity.com/paper/2211.02106/) (2022)
- [Hyperparameter Optimization for SecureBoost via Constrained Multi-Objective Federated Learning](https://mlsanity.com/paper/2404.04490/) (2024)
- [Comparison of Clustering Algorithms for Statistical Features of Vibration Data Sets](https://mlsanity.com/paper/2305.06753/) (2023)
- [FedL2P: Federated Learning to Personalize](https://mlsanity.com/paper/2310.02420/) (2023)

## Abstract (arXiv)

Federated Learning (FL) decouples model training from the need for direct access to the data and allows organizations to collaborate with industry partners to reach a satisfying level of performance without sharing vulnerable business information. The performance of a machine learning algorithm is highly sensitive to the choice of its hyperparameters. In an FL setting, hyperparameter optimization poses new challenges. In this work, we investigated the impact of different hyperparameter optimization approaches in an FL system. In an effort to reduce communication costs, a critical bottleneck in FL, we investigated a local hyperparameter optimization approach that -- in contrast to a global hyperparameter optimization approach -- allows every client to have its own hyperparameter configuration. We implemented these approaches based on grid search and Bayesian optimization and evaluated the algorithms on the MNIST data set using an i.i.d. partition and on an Internet of Things (IoT) sensor based industrial data set using a non-i.i.d. partition.