mlsanityData to 14 Oct 2026

Accelerating Federated Learning via Momentum Gradient Descent

arXiv 1910.03197 ↗First posted 8 Oct 2019cs.LG stat.MLcited by 76

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) provides a communication-efficient approach to solve machine learning problems concerning distributed data, without sending raw data to a central server. However, existing works on FL only utilize first-order gradient descent (GD) and do not consider the preceding iterations to gradient update which can potentially accelerate convergence. In this paper, we consider momentum term which relates to the last iteration. The proposed momentum federated learning (MFL) uses momentum gradient descent (MGD) in the local update step of FL system. We establish global convergence properties of MFL and derive an upper bound on MFL convergence rate. Comparing the upper bounds on MFL and FL convergence rate, we provide conditions in which MFL accelerates the convergence. For different machine learning models, the convergence performance of MFL is evaluated based on experiments with MNIST dataset. Simulation results comfirm that MFL is globally convergent and further reveal significant convergence improvement over FL.

Key facts. Cited by 76 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 19; 2022: 16; 2023: 14; 2024: 17; 2025: 5; 2026: 5.

76 papers.

Sort:

By area:

  1. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  2. 2023Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimizationcited by 65
  3. 2021Decentralized Federated Learning: Balancing Communication and Computing Costscited by 55
  4. 2023Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Modelscited by 50
  5. 2021Gradient-Leakage Resilient Federated Learningcited by 40
  6. 2021FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Modelscited by 31
  7. 2023L-DAWA: Layer-wise Divergence Aware Weight Aggregation in Federated Self-Supervised Visual Representation Learningcited by 30
  8. 2021Implicit Gradient Alignment in Distributed and Federated Learningcited by 29
  9. 2021Cost-Effective Federated Learning in Mobile Edge Networkscited by 25
  10. 2022DELTA: Diverse Client Sampling for Fasting Federated Learningcited by 24
  11. 2021The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learningcited by 23
  12. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23
38 more
  1. 2023AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networkscited by 21
  2. 2023Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Uglycited by 17
  3. 2021A Decentralized Federated Learning Framework via Committee Mechanism with Convergence Guaranteecited by 16
  4. 2025Can Textual Gradient Work in Federated Learning?cited by 15
  5. 2021Semi-Synchronous Federated Learning for Energy-Efficient Training and Accelerated Convergence in Cross-Silo Settingscited by 11
  6. 2022FAIR-FATE: Fair Federated Learning with Momentumcited by 10
  7. 2021Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategiescited by 9
  8. 2022Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Datacited by 9
  9. 2022Federated Data Analytics: A Study on Linear Modelscited by 9
  10. 2023Bridging the Gap Between Foundation Models and Heterogeneous Federated Learningcited by 9
  11. 2021From Deterioration to Acceleration: A Calibration Approach to Rehabilitating Step Asynchronism in Federated Optimizationcited by 8
  12. 2022Federated Progressive Sparsification (Purge, Merge, Tune)+cited by 8
  13. 2023FedHyper: A Universal and Robust Learning Rate Scheduler for Federated Learning with Hypergradient Descentcited by 8
  14. 2022Confederated Learning: Federated Learning with Decentralized Edge Serverscited by 7
  15. 2023Advancements in Federated Learning: Models, Methods, and Privacycited by 7
  16. 2022Gromov-Wasserstein Discrepancy with Local Differential Privacy for Distributed Structural Graphscited by 6
  17. 2022Accelerating Asynchronous Federated Learning Convergence via Opportunistic Mobile Relayingcited by 6
  18. 2022Energy and Spectrum Efficient Federated Learning via High-Precision Over-the-Air Computationcited by 6
  19. 2022Hierarchical Federated Learning with Momentum Acceleration in Multi-Tier Networkscited by 6
  20. 2022Addressing Heterogeneity in Federated Learning via Distributional Transformationcited by 6
  21. 2023Communication-Efficient Design for Quantized Decentralized Federated Learningcited by 6
  22. 2024FedLion: Faster Adaptive Federated Optimization with Fewer Communicationcited by 6
  23. 2024Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Datacited by 6
  24. 2021Expert System Gradient Descent Style Training: Development of a Defensible Artificial Intelligence Techniquecited by 5
  25. 2024Projected Forward Gradient-Guided Frank-Wolfe Algorithm via Variance Reductioncited by 5
  26. 2024Ordered Momentum for Asynchronous SGDcited by 5
  27. 2021Accelerating Federated Learning with a Global Biased Optimisercited by 4
  28. 2022Personalized Federated Learning with Exact Stochastic Gradient Descentcited by 4
  29. 2024Single-Timescale Multi-Sequence Stochastic Approximation Without Fixed Point Smoothness: Theories and Applicationscited by 4
  30. 2021Sample-based and Feature-based Federated Learning for Unconstrained and Constrained Nonconvex Optimization via Mini-batch SSCAcited by 3
  31. 2023A Comparative Study of Federated Learning Models for COVID-19 Detectioncited by 3
  32. 2024An ADRC-Incorporated Stochastic Gradient Descent Algorithm for Latent Factor Analysiscited by 3
  33. 2024Federated and Transfer Learning for Cancer Detection Based on Image Analysiscited by 3
  34. 2024FedUHB: Accelerating Federated Unlearning via Polyak Heavy Ball Methodcited by 3
  35. 2024Review of Mathematical Optimization in Federated Learningcited by 3
  36. 2025An Empirical Study of Federated Prompt Learning for Vision Language Modelcited by 3
  37. 2021Sum-Rate-Distortion Function for Indirect Multiterminal Source Coding in Federated Learningcited by 2
  38. 2021Server Averaging for Federated Learningcited by 2

50 most cited of 76 All 76 (JSON)