mlsanityData to 7 Oct 2026

FedNSAM:Consistency of Local and Global Flatness for Federated Learning

arXiv 2602.23827 ↗First posted 27 Feb 2026cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

In federated learning (FL), multi-step local updates and data heterogeneity usually lead to sharper global minima, which degrades the performance of the global model. Popular FL algorithms integrate sharpness-aware minimization (SAM) into local training to address this issue. However, in the high data heterogeneity setting, the flatness in local training does not imply the flatness of the global model. Therefore, minimizing the sharpness of the local loss surfaces on the client data does not enable the effectiveness of SAM in FL to improve the generalization ability of the global model. We define the flatness distance to explain this phenomenon. By rethinking the SAM in FL and theoretically analyzing the flatness distance, we propose a novel FedNSAM algorithm that accelerates the SAM algorithm by introducing global Nesterov momentum into the local update to harmonize the consistency of global and local flatness. FedNSAM uses the global Nesterov momentum as the direction of local estimation of client global perturbations and extrapolation. Theoretically, we prove a tighter convergence bound than FedSAM by Nesterov extrapolation. Empirically, we conduct comprehensive experiments on CNN and Transformer models to verify the superior performance and efficiency of FedNSAM. The code is available at https://github.com/junkangLiu0/FedNSAM.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 21 of its 34 references that are in the map. First posted on arXiv 27 Feb 2026.

Builds on

21 of 34 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  7. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  8. 2020The Future of Digital Health with Federated Learningcited by 625
  9. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 165
  10. 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
  11. 2022Improving Generalization in Federated Learning by Seeking Flat Minimacited by 98
  12. 2020Differentially Private Secure Multi-Party Computation for Federated Learning in Financial Applicationscited by 58
9 more
  1. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  2. 2024Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimizationcited by 28
  3. 2025FedSWA: Improving Generalization in Federated Learning with Highly Heterogeneous Data via Momentum-Based Stochastic Controlled Weight Averagingcited by 25
  4. 2025FedMuon: Accelerating Federated Learning with Matrix Orthogonalizationcited by 15
  5. 2025FedAdamW: A Communication-Efficient Optimizer with Convergence and Generalization Guarantees for Federated Large Modelscited by 15
  6. 2025From Captions to Rewards (CAREVL): Leveraging Large Language Model Experts for Enhanced Reward Modeling in Large Vision-Language Modelscited by 10
  7. 2025DP-FedPGN: Finding Global Flat Minima for Differentially Private Federated Learning via Penalizing Gradient Normcited by 9
  8. 2025Secure Tug-of-War (SecTOW): Iterative Defense-Attack Training with Reinforcement Learning for Multimodal Model Securitycited by 4
  9. 2026Taming Preconditioner Drift: Unlocking the Potential of Second-Order Optimizers for Federated Learning on Non-IID Datacited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026FedBCD:Communication-Efficient Accelerated Block Coordinate Gradient Descent for Federated Learningcited by 7
  2. 2026Deconstructing the Failure of Ideal Noise Correction: A Three-Pillar Diagnosiscited by 1
  3. 2026SeFoRA: Sketch-Aggregated Federated Low-Rank Adaptation with Heterogeneous Client Rankscited by 1
  4. 2026PDD: Manifold-Prior Diverse Distillation for Medical Anomaly Detectioncited by 0
  5. 2026CTP-FL: Common-Trajectory Gradient Prediction for Federated Learningcited by 0
  6. 2026FedMIX-P: Mixing Local and Global Preconditioners for Federated Vision and Language Model Trainingcited by 0
  7. 2026FedLore: Communication and Memory Efficient Federated Learning via Shared Gradient Low-Rank Projectioncited by 0