mlsanityData to 14 Oct 2026

Generalized Federated Learning via Sharpness Aware Minimization

arXiv 2206.02618 ↗First posted 6 Jun 2022cs.LGcited by 144

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a promising framework for performing privacy-preserving, distributed learning with a set of clients. However, the data distribution among clients often exhibits non-IID, i.e., distribution shift, which makes efficient optimization difficult. To tackle this problem, many FL algorithms focus on mitigating the effects of data heterogeneity across clients by increasing the performance of the global model. However, almost all algorithms leverage Empirical Risk Minimization (ERM) to be the local optimizer, which is easy to make the global model fall into a sharp valley and increase a large deviation of parts of local clients. Therefore, in this paper, we revisit the solutions to the distribution shift problem in FL with a focus on local learning generality. To this end, we propose a general, effective algorithm, textttFedSAM, based on Sharpness Aware Minimization (SAM) local optimizer, and develop a momentum FL algorithm to bridge local and global models, textttMoFedSAM. Theoretically, we show the convergence analysis of these two algorithms and demonstrate the generalization bound of textttFedSAM. Empirically, our proposed algorithms substantially outperform existing FL studies and significantly decrease the learning deviation.

Key facts. Cited by 144 papers in the map (data through 14 Oct 2026). Builds on 42 of its 56 references that are in the map. First posted on arXiv 6 Jun 2022.

Builds on

42 of 56 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  6. 2019Advances and Open Problems in Federated Learningcited by 3,046
  7. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  8. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  9. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  10. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  11. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  12. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
30 more
  1. 2020Adaptive Federated Optimizationcited by 1,046
  2. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  3. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  4. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  5. 2018Federated Learning for Mobile Keyboard Predictioncited by 572
  6. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  7. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 502
  8. 2019Agnostic Federated Learningcited by 456
  9. 2019Fair Resource Allocation in Federated Learningcited by 394
  10. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  11. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 359
  12. 2020Adaptive Personalized Federated Learningcited by 358
  13. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 314
  14. 2020Kernel and Rich Regimes in Overparametrized Modelscited by 306
  15. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  16. 2022Surrogate Gap Minimization Improves Sharpness-Aware Trainingcited by 172
  17. 2021Efficient Sharpness-aware Minimization for Improved Training of Neural Networkscited by 137
  18. 2019SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcited by 121
  19. 2021Local Learning Matters: Rethinking Data Heterogeneity in Federated Learningcited by 114
  20. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  21. 2020Robust Federated Learning: The Case of Affine Distribution Shiftscited by 102
  22. 2022Improving Generalization in Federated Learning by Seeking Flat Minimacited by 98
  23. 2018Universal Adversarial Trainingcited by 87
  24. 2020Fairness-aware Agnostic Federated Learningcited by 87
  25. 2021Federated Reconstruction: Partially Local Federated Learningcited by 79
  26. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  27. 2018Generalizable Adversarial Training via Spectral Normalizationcited by 68
  28. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  29. 2021STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learningcited by 43
  30. 2019The intriguing role of module criticality in the generalization of deep networkscited by 40

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 41; 2024: 38; 2025: 45; 2026: 18.

144 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
  2. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 99
  3. 2022Federated Learning on Non-IID Graphs via Structural Knowledge Sharingcited by 88
  4. 2023FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learningcited by 70
  5. 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
  6. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  7. 2023Improving the Model Consistency of Decentralized Federated Learningcited by 45
  8. 2023An Adaptive Policy to Employ Sharpness-Aware Minimizationcited by 43
  9. 2023Make Landscape Flatter in Differentially Private Federated Learningcited by 33
  10. 2024Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimizationcited by 28
  11. 2024Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boostingcited by 27
  12. 2025FedSWA: Improving Generalization in Federated Learning with Highly Heterogeneous Data via Momentum-Based Stochastic Controlled Weight Averagingcited by 25
38 more
  1. 2023FedFA: Federated Feature Augmentationcited by 22
  2. 2023AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networkscited by 21
  3. 2023Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initializationcited by 21
  4. 2022Beyond Losses Reweighting: Empowering Multi-Task Learning via the Generalization Perspectivecited by 15
  5. 2023Differentially Private Sharpness-Aware Trainingcited by 14
  6. 2023FedSoup: Improving Generalization and Personalization in Federated Learning via Selective Model Interpolationcited by 14
  7. 2023FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learningcited by 14
  8. 2024A Universal Class of Sharpness-Aware Minimization Algorithmscited by 14
  9. 2024FOOGD: Federated Collaboration for Both Out-of-distribution Generalization and Detectioncited by 14
  10. 2024Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcarecited by 12
  11. 2025Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learningcited by 12
  12. 2024FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusioncited by 11
  13. 2023Flat Seeking Bayesian Neural Networkscited by 10
  14. 2023AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learningcited by 10
  15. 2024Exploring Vacant Classes in Label-Skewed Federated Learningcited by 10
  16. 2023ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learningcited by 9
  17. 2024Distributed Federated Learning-Based Deep Learning Model for Privacy MRI Brain Tumor Detectioncited by 9
  18. 2023Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Trainingcited by 8
  19. 2023Federated Learning with Manifold Regularization and Normalized Update Reaggregationcited by 8
  20. 2024Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learningcited by 8
  21. 2023Window-based Model Averaging Improves Generalization in Heterogeneous Federated Learningcited by 7
  22. 2024Federated Motor Imagery Classification for Privacy-Preserving Brain-Computer Interfacescited by 7
  23. 2025Federated Domain Generalization with Data-free On-server Matching Gradientcited by 7
  24. 2025Federated Learning for Diffusion Modelscited by 7
  25. 2025Federated Learning with Domain Shift Erasercited by 7
  26. 2026FedNSAM:Consistency of Local and Global Flatness for Federated Learningcited by 7
  27. 2023DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learningcited by 6
  28. 2023Fake It Till Make It: Federated Learning with Consensus-Oriented Generationcited by 6
  29. 2023Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacycited by 5
  30. 2023Optimal Transport Model Distributional Robustnesscited by 5
  31. 2024From Optimization to Generalization: Fair Federated Learning against Quality Shift via Inter-Client Sharpness Matchingcited by 5
  32. 2024FedCAda: Adaptive Client-Side Optimization for Accelerated and Stable Federated Learningcited by 5
  33. 2024Improving Generalization with Flat Hilbert Bayesian Inferencecited by 5
  34. 2024Navigating Distribution Shifts in Medical Image Analysis: A Surveycited by 5
  35. 2025FAA-CLIP: Federated Adversarial Adaptation of CLIPcited by 5
  36. 2025From Laboratory to Real World: A New Benchmark Towards Privacy-Preserved Visible-Infrared Person Re-Identificationcited by 5
  37. 2023Tackling the Non-IID Issue in Heterogeneous Federated Learning by Gradient Harmonizationcited by 4
  38. 2025Revisit the Stability of Vanilla Federated Learning Under Diverse Conditionscited by 4

50 most cited of 144 All 144 (JSON)