mlsanityData to 7 Oct 2026

Heterogeneous Federated Learning via Grouped Sequential-to-Parallel Training

arXiv 2201.12976 ↗First posted 31 Jan 2022cs.LG cs.AI cs.DCcited by 7

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is a rapidly growing privacy-preserving collaborative machine learning paradigm. In practical FL applications, local data from each data silo reflect local usage patterns. Therefore, there exists heterogeneity of data distributions among data owners (a.k.a. FL clients). If not handled properly, this can lead to model performance degradation. This challenge has inspired the research field of heterogeneous federated learning, which currently remains open. In this paper, we propose a data heterogeneity-robust FL approach, FedGSP, to address this challenge by leveraging on a novel concept of dynamic Sequential-to-Parallel (STP) collaborative training. FedGSP assigns FL clients to homogeneous groups to minimize the overall distribution divergence among groups, and increases the degree of parallelism by reassigning more groups in each round. It is also incorporated with a novel Inter-Cluster Grouping (ICG) algorithm to assist in group assignment, which uses the centroid equivalence theorem to simplify the NP-hard grouping problem to make it solvable. Extensive experiments have been conducted on the non-i.i.d. FEMNIST dataset. The results show that FedGSP improves the accuracy by 3.7% on average compared with seven state-of-the-art approaches, and reduces the training time and communication overhead by more than 90%.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 15 of its 22 references that are in the map. First posted on arXiv 31 Jan 2022.

Builds on

15 of 22 references.

References · 15

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2018Federated Learning with Non-IID Datacited by 1,099
  5. 2020Adaptive Federated Optimizationcited by 1,036
  6. 2018LEAF: A Benchmark for Federated Settingscited by 780
  7. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  8. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  9. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  10. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  11. 2020Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challengescited by 138
  12. 2019Astraea: Self-balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applicationscited by 64
3 more
  1. 2020Inverse Distance Aggregation for Federated Learning with Non-IID Datacited by 44
  2. 2019Hybrid-FL for Wireless Networks: Cooperative Learning Mechanism Using Non-IID Datacited by 38
  3. 2019Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updatingcited by 22

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 4.

7 papers.

  1. 2022MetaFed: Federated Learning among Federations with Cyclic Knowledge Distillation for Personalized Healthcarecited by 36
  2. 2022Data Heterogeneity-Robust Federated Learning via Group Client Selection in Industrial IoTcited by 25
  3. 2023A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacycited by 24
  4. 2022HFedMS: Heterogeneous Federated Learning with Memorable Data Semantics in Industrial Metaversecited by 14
  5. 2023Knowledge-Enhanced Semi-Supervised Federated Learning for Aggregating Heterogeneous Lightweight Clients in IoTcited by 6
  6. 2023StoCFL: A Stochastically Clustered Federated Learning Framework for Non-IID Data with Dynamic Client Participationcited by 4
  7. 2023Communication-Efficient Heterogeneous Federated Learning with Generalized Heavy-Ball Momentumcited by 3