mlsanityData to 7 Oct 2026

Federated Learning over Connected Modes

arXiv 2403.03333 ↗First posted 5 Mar 2024cs.LG cs.DCcited by 3

Not on the map. Why?

Abstract (arXiv)

Statistical heterogeneity in federated learning poses two major challenges: slow global training due to conflicting gradient signals, and the need of personalization for local distributions. In this work, we tackle both challenges by leveraging recent advances in linear mode connectivity -- identifying a linearly connected low-loss region in the parameter space of neural networks, which we call solution simplex. We propose federated learning over connected modes (textscFloco), where clients are assigned local subregions in this simplex based on their gradient signals, and together learn the shared global solution simplex. This allows personalization of the client models to fit their local distributions within the degrees of freedom in the solution simplex and homogenizes the update signals for the global simplex training. Our experiments show that textscFloco accelerates the global training process, and significantly improves the local accuracy with minimal computational overhead in cross-silo federated learning settings.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 42 of its 58 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

42 of 58 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2018Federated Learning with Non-IID Datacited by 1,099
  7. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  8. 2020Adaptive Federated Optimizationcited by 1,036
  9. 2018LEAF: A Benchmark for Federated Settingscited by 780
  10. 2017Federated Multi-Task Learningcited by 669
  11. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  12. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 627
30 more
  1. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  2. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  3. 2019Federated Learning with Personalization Layerscited by 533
  4. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  5. 2021Towards Personalized Federated Learningcited by 462
  6. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
  7. 2019Fair Resource Allocation in Federated Learningcited by 389
  8. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  9. 2020Adaptive Personalized Federated Learningcited by 353
  10. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  11. 2018Essentially No Barriers in Neural Network Energy Landscapecited by 328
  12. 2019Robust Aggregation for Federated Learningcited by 314
  13. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  14. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 223
  15. 2015Convergent Learning: Do different neural networks learn the same representations?cited by 215
  16. 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
  17. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  18. 2020Survey of Personalization Techniques for Federated Learningcited by 144
  19. 2020Linear Mode Connectivity in Multitask and Continual Learningcited by 144
  20. 2016Topology and Geometry of Half-Rectified Network Optimizationcited by 102
  21. 2021Learning Neural Network Subspacescited by 82
  22. 2021Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensemblingcited by 75
  23. 2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
  24. 2022Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learningcited by 71
  25. 2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 67
  26. 2021Fed-ensemble: Improving Generalization through Model Ensembling in Federated Learningcited by 23
  27. 2023Federated Learning under Heterogeneous and Correlated Client Availabilitycited by 18
  28. 2022DORA: Exploring Outlier Representations in Deep Neural Networkscited by 14
  29. 2021Connecting Low-Loss Subspace for Personalized Federated Learningcited by 12
  30. 2023FedZero: Leveraging Renewable Excess Energy in Federated Learningcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Aggregation on Learnable Manifolds for Asynchronous Federated Optimizationcited by 1
  2. 2026Heterogeneity-Aware Client Selection Methodology For Efficient Federated Learningcited by 0
  3. 2026FedOBP: Federated Optimal Brain Personalization through Cloud-Edge Element-wise Decouplingcited by 0