mlsanityData to 14 Oct 2026

Federated Learning Hyper-Parameter Tuning from a System Perspective

arXiv 2211.13656 ↗First posted 24 Nov 2022cs.LG cs.DCcited by 1

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is a distributed model training paradigm that preserves clients' data privacy. It has gained tremendous attention from both academia and industry. FL hyper-parameters (e.g., the number of selected clients and the number of training passes) significantly affect the training overhead in terms of computation time, transmission time, computation load, and transmission load. However, the current practice of manually selecting FL hyper-parameters imposes a heavy burden on FL practitioners because applications have different training preferences. In this paper, we propose FedTune, an automatic FL hyper-parameter tuning algorithm tailored to applications' diverse system requirements in FL training. FedTune iteratively adjusts FL hyper-parameters during FL training and can be easily integrated into existing FL systems. Through extensive evaluations of FedTune for diverse applications and FL aggregation algorithms, we show that FedTune is lightweight and effective, achieving 8.48%-26.75% system overhead reduction compared to using fixed FL hyper-parameters. This paper assists FL practitioners in designing high-performance FL training solutions. The source code of FedTune is available at https://github.com/DataSysTech/FedTune.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 52 references that are in the map. First posted on arXiv 24 Nov 2022.

Builds on

27 of 52 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  5. 2020Adaptive Federated Optimizationcited by 1,046
  6. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  7. 2019Towards Federated Learning at Scale: System Designcited by 832
  8. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  9. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 699
  10. 2018Federated Learning for Mobile Keyboard Predictioncited by 572
  11. 2021A Field Guide to Federated Optimizationcited by 315
  12. 2019A Survey on Distributed Machine Learningcited by 188
15 more
  1. 2020On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practicecited by 182
  2. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 136
  3. 2021Federated Evaluation and Tuning for On-Device Personalization: System Design & Applicationscited by 85
  4. 2020Federated Bayesian Optimization via Thompson Samplingcited by 77
  5. 2021Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharingcited by 58
  6. 2021Federated Learning for Internet of Things: Applications, Challenges, and Opportunitiescited by 50
  7. 2021Differentially Private Federated Bayesian Optimization with Distributed Explorationcited by 32
  8. 2019Robust Federated Learning Through Representation Matching and Adaptive Hyper-parameterscited by 16
  9. 2022Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentationcited by 16
  10. 2021FLoRA: Single-shot Hyper-parameter Optimization for Federated Learningcited by 14
  11. 2020Privacy-Preserving Technology to Help Millions of People: Federated Prediction Model for Stroke Preventioncited by 11
  12. 2019Intelligent Traffic Monitoring Systems for Vehicle Classification: A Surveycited by 10
  13. 2020Deep Learning in the Era of Edge Computing: Challenges and Opportunitiescited by 10
  14. 2021FedTune: Automatic Tuning of Federated Learning Hyper-Parameters from System Perspectivecited by 7
  15. 2022FedHPO-B: A Benchmark Suite for Federated Hyperparameter Optimizationcited by 6

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024CAFe: Cost and Age aware Federated Learningcited by 0