mlsanityData to 7 Oct 2026

Not All Federated Learning Algorithms Are Created Equal: A Performance Evaluation Study

arXiv 2403.17287 ↗First posted 26 Mar 2024cs.LG cs.DCcited by 10

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) emerged as a practical approach to training a model from decentralized data. The proliferation of FL led to the development of numerous FL algorithms and mechanisms. Many prior efforts have given their primary focus on accuracy of those approaches, but there exists little understanding of other aspects such as computational overheads, performance and training stability, etc. To bridge this gap, we conduct extensive performance evaluation on several canonical FL algorithms (FedAvg, FedProx, FedYogi, FedAdam, SCAFFOLD, and FedDyn) by leveraging an open-source federated learning framework called Flame. Our comprehensive measurement study reveals that no single algorithm works best across different performance metrics. A few key observations are: (1) While some state-of-the-art algorithms achieve higher accuracy than others, they incur either higher computation overheads (FedDyn) or communication overheads (SCAFFOLD). (2) Recent algorithms present smaller standard deviation in accuracy across clients than FedAvg, indicating that the advanced algorithms' performances are stable. (3) However, algorithms such as FedDyn and SCAFFOLD are more prone to catastrophic failures without the support of additional techniques such as gradient clipping. We hope that our empirical study can help the community to build best practices in evaluating FL algorithms.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 19 of its 27 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

19 of 27 references.

References · 19

  1. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  2. 2020Adaptive Federated Optimizationcited by 1,036
  3. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  4. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
  5. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  6. 2019Fair Resource Allocation in Federated Learningcited by 389
  7. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
  8. 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 335
  9. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  10. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  11. 2021FairFed: Enabling Group Fairness in Federated Learningcited by 114
  12. 2019Hierarchical Federated Learning Across Heterogeneous Cellular Networkscited by 98
7 more
  1. 2021On Large-Cohort Training for Federated Learningcited by 73
  2. 2020Mitigating Bias in Federated Learningcited by 56
  3. 2022FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clientscited by 46
  4. 2021Training Fair Models in Federated Learning without Data Privacy Infringementcited by 26
  5. 2021Resource-Efficient Federated Learningcited by 26
  6. 2023Mitigating Group Bias in Federated Learning: Beyond Local Fairnesscited by 10
  7. 2023Flame: Simplifying Topology Extension in Federated Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 3.

10 papers.

  1. 2025DP-FedLoRA: Privacy-Enhanced Federated Fine-Tuning for On-Device Large Language Modelscited by 4
  2. 2025CLLoRA: An Approach to Measure the Effects of the Context Length for LLM Fine-Tuningcited by 2
  3. 2025GC-Fed: Gradient Centralized Federated Learning with Partial Client Participationcited by 2
  4. 2025SAFE: Self-Adjustment Federated Learning Framework for Remote Sensing Collaborative Perceptioncited by 1
  5. 2026Self-Play Enhancement via Advantage-Weighted Refinement in Online Federated LLM Fine-Tuning with Real-Time Feedbackcited by 1
  6. 2024Where is the Testbed for my Federated Learning Research?cited by 0
  7. 2025FedVSR: Towards Model-Agnostic Federated Learning in Video Super-Resolutioncited by 0
  8. 2025SimDeep: Federated 3D Indoor Localization via Similarity-Aware Aggregationcited by 0
  9. 2026How Edge of Stability Hinders SCAFFOLD in Federated Optimizationcited by 0
  10. 2026Pushing the (Decision) Boundaries: Dynamically Calibrating Differentially Private Noise to Explainability in Federated Learningcited by 0