mlsanityData to 7 Oct 2026

Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning

arXiv 2412.03752 ↗First posted 4 Dec 2024cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) enables collaborative model training with privacy preservation. Data heterogeneity across edge devices (clients) can cause models to converge to sharp minima, negatively impacting generalization and robustness. Recent approaches use client-side sharpness-aware minimization (SAM) to encourage flatter minima, but the discrepancy between local and global loss landscapes often undermines their effectiveness, as optimizing for local sharpness does not ensure global flatness. This work introduces FedGloSS (Federated Global Server-side Sharpness), a novel FL approach that prioritizes the optimization of global sharpness on the server, using SAM. To reduce communication overhead, FedGloSS cleverly approximates sharpness using the previous global gradient, eliminating the need for additional client communication. Our extensive evaluations demonstrate that FedGloSS consistently reaches flatter minima and better performance compared to state-of-the-art FL methods across various federated vision benchmarks.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 45 of its 62 references that are in the map. First posted on arXiv 4 Dec 2024.

Builds on

45 of 62 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2019Advances and Open Problems in Federated Learningcited by 3,021
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  8. 2018Group Normalizationcited by 1,514
  9. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  10. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  11. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  12. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
33 more
  1. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  2. 2020Adaptive Federated Optimizationcited by 1,036
  3. 2016An overview of gradient descent optimization algorithmscited by 1,021
  4. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  5. 2018LEAF: A Benchmark for Federated Settingscited by 780
  6. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  7. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  8. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  9. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
  10. 2020Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrievalcited by 305
  11. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
  12. 2021FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Spacecited by 284
  13. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  14. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 152
  15. 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
  16. 2021Efficient Sharpness-aware Minimization for Improved Training of Neural Networkscited by 137
  17. 2022Towards Efficient and Scalable Sharpness-Aware Minimizationcited by 129
  18. 2020Federated Visual Classification with Real-World Data Distributioncited by 126
  19. 2022Improving Generalization in Federated Learning by Seeking Flat Minimacited by 98
  20. 2022Sharpness-Aware Training for Freecited by 95
  21. 2021Federated Learning for Internet of Things: Applications, Challenges, and Opportunitiescited by 49
  22. 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
  23. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  24. 2022FedDrive: Generalizing Federated Learning to Semantic Segmentation in Autonomous Drivingcited by 41
  25. 2022Learning Across Domains and Devices: Style-Driven Source-Free Domain Adaptation in Clustered Federated Learningcited by 35
  26. 2022Federated Learning for Medical Applications: A Taxonomy, Current Trends, Challenges, and Future Research Directionscited by 34
  27. 2023Make Landscape Flatter in Differentially Private Federated Learningcited by 33
  28. 2021FedJAX: Federated learning simulation with JAXcited by 32
  29. 2017Accelerated Stochastic Power Iterationcited by 22
  30. 2022AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimationcited by 21
  31. 2023Differentially Private Sharpness-Aware Trainingcited by 14
  32. 2023Window-based Model Averaging Improves Generalization in Heterogeneous Federated Learningcited by 7
  33. 2023Communication-Efficient Heterogeneous Federated Learning with Generalized Heavy-Ball Momentumcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 5.

8 papers.

  1. 2025Distributed Zero-Shot Learning for Visual Recognitioncited by 3
  2. 2026Subspace Optimization for Efficient Federated Learning under Heterogeneous Datacited by 1
  3. 2026Trustworthy Federated Label Distribution Learning under Annotation Quality Disparitycited by 1
  4. 2025LSAM: Asynchronous Distributed Training with Landscape-Smoothed Sharpness-Aware Minimizationcited by 0
  5. 2025Cooperative Pseudo Labeling for Unsupervised Federated Classificationcited by 0
  6. 2026Personalized Federated Learning for Gradient Alignmentcited by 0
  7. 2026FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learningcited by 0
  8. 2026FedFFT: Taming Client Drift in Federated SAM via Spectral Perturbation Filteringcited by 0