# FedTune: Automatic Tuning of Federated Learning Hyper-Parameters from System Perspective

arXiv: https://arxiv.org/abs/2110.03061 · HTML: https://mlsanity.com/paper/2110.03061/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 6 Oct 2021.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [A Survey on Distributed Machine Learning](https://mlsanity.com/paper/1912.09789/) (2019)
- [On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice](https://mlsanity.com/paper/2007.15745/) (2020)
- [FedScale: Benchmarking Model and System Performance of Federated Learning at Scale](https://mlsanity.com/paper/2105.11367/) (2021)
- [Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications](https://mlsanity.com/paper/2102.08503/) (2021)
- [Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing](https://mlsanity.com/paper/2106.04502/) (2021)
- [Intelligent Traffic Monitoring Systems for Vehicle Classification: A Survey](https://mlsanity.com/paper/1910.04656/) (2019)
- [Deep Learning in the Era of Edge Computing: Challenges and Opportunities](https://mlsanity.com/paper/2010.08861/) (2020)

## Built on by (most cited first, 7 of 7)

- [Client Selection in Federated Learning: Principles, Challenges, and Opportunities](https://mlsanity.com/paper/2211.01549/) (2022)
- [FederatedScope-GNN: Towards a Unified, Comprehensive and Efficient Package for Federated Graph Learning](https://mlsanity.com/paper/2204.05562/) (2022)
- [On Noisy Evaluation in Federated Hyperparameter Tuning](https://mlsanity.com/paper/2212.08930/) (2022)
- [FedHPO-B: A Benchmark Suite for Federated Hyperparameter Optimization](https://mlsanity.com/paper/2206.03966/) (2022)
- [Federated Learning Hyper-Parameter Tuning from a System Perspective](https://mlsanity.com/paper/2211.13656/) (2022)
- [FedPop: Federated Population-based Hyperparameter Tuning](https://mlsanity.com/paper/2308.08634/) (2023)
- [DistDD: Distributed Data Distillation Aggregation through Gradient Matching](https://mlsanity.com/paper/2410.08665/) (2024)

## Abstract (arXiv)

Federated learning (FL) hyper-parameters significantly affect the training overheads in terms of computation time, transmission time, computation load, and transmission load. However, the current practice of manually selecting FL hyper-parameters puts a high burden on FL practitioners since various applications prefer different training preferences. In this paper, we propose FedTune, an automatic FL hyper-parameter tuning algorithm tailored to applications' diverse system requirements of FL training. FedTune is lightweight and flexible, achieving 8.48%-26.75% improvement for different datasets compared to fixed FL hyper-parameters.