mlsanityData to 14 Oct 2026

Federated Geometric Monte Carlo Clustering to Counter Non-IID Datasets

arXiv 2204.11017 ↗First posted 23 Apr 2022cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated learning allows clients to collaboratively train models on datasets that are acquired in different locations and that cannot be exchanged because of their size or regulations. Such collected data is increasingly non-independent and non-identically distributed (non-IID), negatively affecting training accuracy. Previous works tried to mitigate the effects of non-IID datasets on training accuracy, focusing mainly on non-IID labels, however practical datasets often also contain non-IID features. To address both non-IID labels and features, we propose FedGMCC, a novel framework where a central server aggregates client models that it can cluster together. FedGMCC clustering relies on a Monte Carlo procedure that samples the output space of client models, infers their position in the weight space on a loss manifold and computes their geometric connection via an affine curve parametrization. FedGMCC aggregates connected models along their path connectivity to produce a richer global model, incorporating knowledge of all connected client models. FedGMCC outperforms FedAvg and FedProx in terms of convergence rates on the EMNIST62 and a genomic sequence classification datasets (by up to +63%). FedGMCC yields an improved accuracy (+4%) on the genomic dataset with respect to CFL, in high non-IID feature space settings and label incongruency.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 25 references that are in the map. First posted on arXiv 23 Apr 2022.

Builds on

18 of 25 references.

References · 18

  1. 2019Advances and Open Problems in Federated Learningcited by 3,046
  2. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  3. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  4. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  5. 2018Federated Learning with Non-IID Datacited by 1,108
  6. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 588
  7. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  8. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  9. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  10. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  11. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 305
  12. 2014The Loss Surfaces of Multilayer Networkscited by 280
6 more
  1. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 223
  2. 2020Linear Mode Connectivity in Multitask and Continual Learningcited by 145
  3. 2016Topology and Geometry of Half-Rectified Network Optimizationcited by 102
  4. 2021Federated Learning with Fair Averagingcited by 61
  5. 2020FedFMC: Sequential Efficient Federated Learning on Non-iid Datacited by 17
  6. 2021H-FL: A Hierarchical Communication-Efficient and Privacy-Protected Architecture for Federated Learningcited by 14

Built on by

Not cited yet.