mlsanityData to 14 Oct 2026

FedCM: Federated Learning with Client-level Momentum

arXiv 2106.10874 ↗First posted 21 Jun 2021cs.LGcited by 77

Not on the map. Why?

Abstract (arXiv)

Federated Learning is a distributed machine learning approach which enables model training without data sharing. In this paper, we propose a new federated learning algorithm, Federated Averaging with Client-level Momentum (FedCM), to tackle problems of partial participation and client heterogeneity in real-world federated learning applications. FedCM aggregates global gradient information in previous communication rounds and modifies client gradient descent with a momentum-like term, which can effectively correct the bias and improve the stability of local SGD. We provide theoretical analysis to highlight the benefits of FedCM. We also perform extensive empirical studies and demonstrate that FedCM achieves superior performance in various tasks and is robust to different levels of client numbers, participation rate and client heterogeneity.

Key facts. Cited by 77 papers in the map (data through 14 Oct 2026). Builds on 33 of its 42 references that are in the map. First posted on arXiv 21 Jun 2021.

Builds on

33 of 42 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  7. 2018Group Normalizationcited by 1,524
  8. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  9. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  10. 2018Federated Learning with Non-IID Datacited by 1,108
  11. 2020Adaptive Federated Optimizationcited by 1,046
  12. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
21 more
  1. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  2. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  3. 2018Local SGD Converges Fast and Communicates Littlecited by 605
  4. 2014Convex Optimization: Algorithms and Complexitycited by 575
  5. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  6. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 305
  7. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  8. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 154
  9. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 144
  10. 2013Communication Efficient Distributed Optimization using an Approximate Newton-type Methodcited by 132
  11. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 123
  12. 2019SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcited by 121
  13. 2020FedSplit: An algorithmic framework for fast federated optimizationcited by 108
  14. 2020Federated Accelerated Stochastic Gradient Descentcited by 85
  15. 2019Variance Reduced Local SGD with Lower Communication Complexitycited by 80
  16. 2019Accelerating Federated Learning via Momentum Gradient Descentcited by 76
  17. 2020FedDANE: A Federated Newton-Type Methodcited by 76
  18. 2021Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Datacited by 69
  19. 2019Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Ratescited by 26
  20. 2020Faster On-Device Training Using New Federated Momentum Algorithmcited by 25
  21. 2020Effective Federated Adaptive Gradient Methods with Non-IID Decentralized Datacited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 7; 2023: 23; 2024: 19; 2025: 16; 2026: 10.

77 papers.

Sort:

By area:

  1. 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
  2. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 92
  3. 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
  4. 2023Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscapecited by 46
  5. 2023Momentum Benefits Non-IID Federated Learning Simply and Provablycited by 38
  6. 2022FedBR: Improving Federated Learning on Heterogeneous Data via Local Learning Bias Reductioncited by 35
  7. 2022Multi-Level Branched Regularization for Federated Learningcited by 34
  8. 2024Relaxed Contrastive Learning for Federated Learningcited by 32
  9. 2022Achieving Personalized Federated Learning with Sparse Local Modelscited by 31
  10. 2024Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimizationcited by 28
  11. 2023GradMA: A Gradient-Memory-based Accelerated Federated Learning with Alleviated Catastrophic Forgettingcited by 27
  12. 2022Communication-Efficient Federated Learning with Accelerated Client Gradientcited by 24
38 more
  1. 2023Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedupcited by 23
  2. 2022Gradient Masked Averaging for Federated Learningcited by 22
  3. 2023Understanding How Consistency Works in Federated Learning via Stage-wise Relaxed Initializationcited by 21
  4. 2025FedMuon: Accelerating Federated Learning with Matrix Orthogonalizationcited by 16
  5. 2025FedAdamW: A Communication-Efficient Optimizer with Convergence and Generalization Guarantees for Federated Large Modelscited by 15
  6. 2023Fusion of Global and Local Knowledge for Personalized Federated Learningcited by 14
  7. 2021FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximationcited by 10
  8. 2024Accelerating Heterogeneous Federated Learning with Closed-form Classifierscited by 10
  9. 2023Combating Data Imbalances in Federated Semi-supervised Learning with Dual Regulatorscited by 9
  10. 2024Tackling Data Heterogeneity in Federated Learning via Loss Decompositioncited by 9
  11. 2023Federated Learning with Manifold Regularization and Normalized Update Reaggregationcited by 8
  12. 2023Window-based Model Averaging Improves Generalization in Heterogeneous Federated Learningcited by 7
  13. 2023FilFL: Client Filtering for Optimized Client Participation in Federated Learningcited by 5
  14. 2023Asymmetrically Decentralized Federated Learningcited by 5
  15. 2023Federated Learning on Non-iid Data via Local and Global Distillationcited by 4
  16. 2025Revisit the Stability of Vanilla Federated Learning Under Diverse Conditionscited by 4
  17. 2026Taming Preconditioner Drift: Unlocking the Potential of Second-Order Optimizers for Federated Learning on Non-IID Datacited by 4
  18. 2022FedSSO: A Federated Server-Side Second-Order Optimization Algorithmcited by 3
  19. 2023Stabilizing and Improving Federated Learning with Non-IID Data and Client Dropoutcited by 3
  20. 2023Cooperative Hardware-Prompt Learning for Snapshot Compressive Imagingcited by 3
  21. 2023FedBug: A Bottom-Up Gradual Unfreezing Framework for Federated Learningcited by 3
  22. 2023Towards Understanding Generalization and Stability Gaps between Centralized and Decentralized Federated Learningcited by 3
  23. 2023Communication-Efficient Heterogeneous Federated Learning with Generalized Heavy-Ball Momentumcited by 3
  24. 2024Momentum Approximation in Asynchronous Private Federated Learningcited by 3
  25. 2024Review of Mathematical Optimization in Federated Learningcited by 3
  26. 2025Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participationcited by 3
  27. 2021LoSAC: An Efficient Local Stochastic Average Control Method for Federated Optimizationcited by 2
  28. 2023FedDA: Faster Framework of Local Adaptive Gradient Methods via Restarted Dual Averagingcited by 2
  29. 2023DualFL: A Duality-based Federated Learning Algorithm with Communication Acceleration in the General Convex Regimecited by 2
  30. 2023FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Datacited by 2
  31. 2024Cohort Squeeze: Beyond a Single Communication Round per Cohort in Cross-Device Federated Learningcited by 2
  32. 2024FedPylot: Navigating Federated Learning for Real-Time Object Detection in Internet of Vehiclescited by 2
  33. 2024A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needscited by 2
  34. 2024Boosting the Performance of Decentralized Federated Learning via Catalyst Accelerationcited by 2
  35. 2024Learnable Sparse Customization in Heterogeneous Edge Computingcited by 2
  36. 2024FedMeld: A Model-dispersal Federated Learning Framework for Space-ground Integrated Networkscited by 2
  37. 2023Federated Learning on Edge Sensing Devices: A Reviewcited by 1
  38. 2024OledFL: Unleashing the Potential of Decentralized Federated Learning via Opposite Lookahead Enhancementcited by 1

50 most cited of 77 All 77 (JSON)