mlsanityData to 13 Oct 2026

FLoRA: Single-shot Hyper-parameter Optimization for Federated Learning

arXiv 2112.08524 ↗First posted 15 Dec 2021cs.LG cs.DCcited by 14

Not on the map. Why?

Abstract (arXiv)

We address the relatively unexplored problem of hyper-parameter optimization (HPO) for federated learning (FL-HPO). We introduce Federated Loss suRface Aggregation (FLoRA), the first FL-HPO solution framework that can address use cases of tabular data and gradient boosting training algorithms in addition to stochastic gradient descent/neural networks commonly addressed in the FL literature. The framework enables single-shot FL-HPO, by first identifying a good set of hyper-parameters that are used in a **single** FL training. Thus, it enables FL-HPO solutions with minimal additional communication overhead compared to FL training without HPO. Our empirical evaluation of FLoRA for Gradient Boosted Decision Trees on seven OpenML data sets demonstrates significant model accuracy improvements over the considered baseline, and robustness to increasing number of parties involved in FL-HPO training.

Key facts. Cited by 14 papers in the map (data through 13 Oct 2026). Builds on 31 of its 50 references that are in the map. First posted on arXiv 15 Dec 2021.

Builds on

31 of 50 references.

References · 31

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,653
  2. 2019Advances and Open Problems in Federated Learningcited by 3,040
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,981
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,470
  5. 2018Federated Learning with Non-IID Datacited by 1,106
  6. 2020Adaptive Federated Optimizationcited by 1,041
  7. 2018On First-Order Meta-Learning Algorithmscited by 1,010
  8. 2019Towards Federated Learning at Scale: System Designcited by 830
  9. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 693
  10. 2017Federated Multi-Task Learningcited by 670
  11. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 668
  12. 2014OpenML: networked science in machine learningcited by 524
19 more
  1. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 467
  2. 2019Agnostic Federated Learningcited by 453
  3. 2018Meta-Learning: A Surveycited by 254
  4. 2015Non-stochastic Best Arm Identification and Hyperparameter Optimizationcited by 221
  5. 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 210
  6. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 175
  7. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 158
  8. 2020TiFL: A Tier-based Federated Learning Systemcited by 96
  9. 2020Federated Bayesian Optimization via Thompson Samplingcited by 77
  10. 2021Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharingcited by 58
  11. 2020IBM Federated Learning: an Enterprise Framework White Paper V0.1cited by 57
  12. 2020Mitigating Bias in Federated Learningcited by 56
  13. 2019HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learningcited by 51
  14. 2019Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learningcited by 50
  15. 2019An ADMM Based Framework for AutoML Pipeline Configurationcited by 47
  16. 2019Robust Federated Learning Through Representation Matching and Adaptive Hyper-parameterscited by 16
  17. 2015Selecting Near-Optimal Learners via Incremental Data Allocationcited by 12
  18. 2020Adaptive Histogram-Based Gradient Boosted Trees for Federated Learningcited by 10
  19. 2018Learning Rate Adaptation for Federated and Differentially Private Learningcited by 7

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 7; 2024: 2.

14 papers.

Sort:

  1. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 149
  2. 2022Federated Neural Banditscited by 17
  3. 2023Hyperparameter Optimization through Neural Network Partitioningcited by 8
  4. 2023FedHyper: A Universal and Robust Learning Rate Scheduler for Federated Learning with Hypergradient Descentcited by 8
  5. 2022FedHPO-B: A Benchmark Suite for Federated Hyperparameter Optimizationcited by 6
  6. 2022On Noisy Evaluation in Federated Hyperparameter Tuningcited by 6
  7. 2022FEATHERS: Federated Architecture and Hyperparameter Searchcited by 3
  8. 2023FedAVO: Improving Communication Efficiency in Federated Learning with African Vultures Optimizercited by 2
  9. 2022Federated Learning Hyper-Parameter Tuning from a System Perspectivecited by 1
  10. 2023FedPop: Federated Population-based Hyperparameter Tuningcited by 1
  11. 2023SODA: Robust Training of Test-Time Data Adaptorscited by 1
  12. 2024Hyper-parameter Optimization for Federated Learning with Step-wise Adaptive Mechanismcited by 1
2 more
  1. 2023Toward Theoretical Guidance for Two Common Questions in Practical Cross-Validation based Hyperparameter Selectioncited by 0
  2. 2024DistDD: Distributed Data Distillation Aggregation through Gradient Matchingcited by 0