mlsanityData to 7 Oct 2026

FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy

arXiv 2302.10429 ↗First posted 21 Feb 2023cs.LG cs.DC math.OCcited by 49

Not on the map. Why?

Abstract (arXiv)

Federated learning is an emerging distributed machine learning framework which jointly trains a global model via a large number of local devices with data privacy protections. Its performance suffers from the non-vanishing biases introduced by the local inconsistent optimal and the rugged client-drifts by the local over-fitting. In this paper, we propose a novel and practical method, FedSpeed, to alleviate the negative impacts posed by these problems. Concretely, FedSpeed applies the prox-correction term on the current local updates to efficiently reduce the biases introduced by the prox-term, a necessary regularizer to maintain the strong local consistency. Furthermore, FedSpeed merges the vanilla stochastic gradient with a perturbation computed from an extra gradient ascent step in the neighborhood, thereby alleviating the issue of local over-fitting. Our theoretical analysis indicates that the convergence rate is related to both the communication rounds T and local intervals K with a upper bound small O(1/T) if setting a proper local interval. Moreover, we conduct extensive experiments on the real-world dataset to demonstrate the efficiency of our proposed FedSpeed, which performs significantly faster and achieves the state-of-the-art (SOTA) performance on the general FL experimental settings than several baselines. Our code is available at urlhttps://github.com/woodenchild95/FL-Simulator.git.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 46 of its 56 references that are in the map. First posted on arXiv 21 Feb 2023.

Builds on

46 of 56 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2018Group Normalizationcited by 1,514
  6. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  7. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  8. 2020Adaptive Federated Optimizationcited by 1,036
  9. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  10. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  11. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  12. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
34 more
  1. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
  2. 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
  3. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  4. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  5. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 178
  6. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  7. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  8. 2022Towards Understanding Sharpness-Aware Minimizationcited by 145
  9. 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
  10. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  11. 2022Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learningcited by 122
  12. 2019SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcited by 121
  13. 2022Improving Generalization in Federated Learning by Seeking Flat Minimacited by 98
  14. 2019First Analysis of Local GD on Heterogeneous Datacited by 80
  15. 2019Variance Reduced Local SGD with Lower Communication Complexitycited by 80
  16. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  17. 2020FedDANE: A Federated Newton-Type Methodcited by 76
  18. 2020From Local SGD to Local Fixed-Point Methods for Federated Learningcited by 74
  19. 2022Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approachcited by 70
  20. 2021From Distributed Machine Learning to Federated Learning: A Surveycited by 65
  21. 2021Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learningcited by 52
  22. 2023Improving the Model Consistency of Decentralized Federated Learningcited by 45
  23. 2022Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Modelscited by 42
  24. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 35
  25. 2021Local Adaptivity in Federated Learning: Convergence and Consistencycited by 31
  26. 2020Quantized Adam with Error Feedbackcited by 27
  27. 2020FedADC: Accelerated Federated Learning with Drift Controlcited by 26
  28. 2021Toward Communication Efficient Adaptive Gradient Methodcited by 25
  29. 2022FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneitycited by 25
  30. 2022Over-the-Air Federated Learning via Second-Order Optimizationcited by 11
  31. 2022Efficient-Adam: Communication-Efficient Distributed Adamcited by 11
  32. 2023Enhance Local Consistency in Federated Learning: A Multi-Step Inertial Momentum Approachcited by 11
  33. 2020Exploiting the Surrogate Gap in Online Multiclass Classificationcited by 9
  34. 2021On Second-order Optimization Methods for Federated Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 17; 2024: 16; 2025: 11; 2026: 5.

49 papers.

Sort:

By area:

  1. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 113
  2. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 89
  3. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 86
  4. 2023Enhancing Fine-Tuning Based Backdoor Defense with Sharpness-Aware Minimizationcited by 60
  5. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  6. 2023Make Landscape Flatter in Differentially Private Federated Learningcited by 33
  7. 2023Robust Generalization against Photon-Limited Corruptions via Worst-Case Sharpness Minimizationcited by 29
  8. 2024Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimizationcited by 28
  9. 2025FedSWA: Improving Generalization in Federated Learning with Highly Heterogeneous Data via Momentum-Based Stochastic Controlled Weight Averagingcited by 25
  10. 2023AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networkscited by 21
  11. 2023Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initializationcited by 21
  12. 2024FedImpro: Measuring and Improving Client Update in Federated Learningcited by 15
37 more
  1. 2023FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learningcited by 14
  2. 2023Efficient Federated Prompt Tuning for Black-box Large Pre-trained Modelscited by 12
  3. 2024FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusioncited by 11
  4. 2023Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Trainingcited by 8
  5. 2024Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learningcited by 8
  6. 2023Window-based Model Averaging Improves Generalization in Heterogeneous Federated Learningcited by 7
  7. 2025Federated Learning with Domain Shift Erasercited by 7
  8. 2023DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learningcited by 6
  9. 2023Distributed Personalized Empirical Risk Minimizationcited by 6
  10. 2025Federated Learning for Diffusion Modelscited by 6
  11. 2023Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacycited by 5
  12. 2023Asymmetrically Decentralized Federated Learningcited by 5
  13. 2024Local Superior Soups: A Catalyst for Model Merging in Cross-Silo Federated Learningcited by 5
  14. 2025Revisit the Stability of Vanilla Federated Learning Under Diverse Conditionscited by 4
  15. 2023Towards Understanding Generalization and Stability Gaps between Centralized and Decentralized Federated Learningcited by 3
  16. 2024Towards Layer-Wise Personalized Federated Learning: Adaptive Layer Disentanglement via Conflicting Gradientscited by 3
  17. 2025A Unified Benchmark of Federated Learning with Kolmogorov-Arnold Networks for Medical Imagingcited by 3
  18. 2025Debunking Optimization Myths in Federated Learning for Medical Image Classificationcited by 3
  19. 2023Regret-Optimal Federated Transfer Learning for Kernel Regression with Applications in American Option Pricingcited by 2
  20. 2024A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needscited by 2
  21. 2024Boosting the Performance of Decentralized Federated Learning via Catalyst Accelerationcited by 2
  22. 2023Adaptive Self-Distillation for Minimizing Client Drift in Heterogeneous Federated Learningcited by 1
  23. 2024Cohort Squeeze: Beyond a Single Communication Round per Cohort in Cross-Device Federated Learningcited by 1
  24. 2024OledFL: Unleashing the Potential of Decentralized Federated Learning via Opposite Lookahead Enhancementcited by 1
  25. 2025When to Stop Federated Learning: Zero-Shot Generation of Synthetic Validation Data with Generative AI for Early Stoppingcited by 1
  26. 2025Minimizing Layerwise Activation Norm Improves Generalization in Federated Learningcited by 1
  27. 2024Neighborhood and Global Perturbations Supported SAM in Federated Learning: From Local Tweaks To Global Awarenesscited by 0
  28. 2024Convergent Differential Privacy Analysis for General Federated Learningcited by 0
  29. 2024Embedding Byzantine Fault Tolerance into Federated Learning via Consistency Scoringcited by 0
  30. 2025Prompt Estimation from Prototypes for Federated Prompt Tuning of Vision Transformerscited by 0
  31. 2025FedSCAl: Leveraging Server and Client Alignment for Unsupervised Federated Source-Free Domain Adaptationcited by 0
  32. 2025FedDPC : Handling Data Heterogeneity and Partial Client Participation in Federated Learningcited by 0
  33. 2026Beyond Fixed Rounds: Data-Free Early Stopping for Practical Federated Learningcited by 0
  34. 2026Rethinking the Personalized Relaxed Initialization in the Federated Learning: Consistency and Generalizationcited by 0
  35. 2026FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learningcited by 0
  36. 2026FedFFT: Taming Client Drift in Federated SAM via Spectral Perturbation Filteringcited by 0
  37. 2026FL-OA: A Byzantine-Robust Federated Learning Framework with Outsourced Auditing for Intelligent Devicescited by 0