# Accelerating Federated Learning via Momentum Gradient Descent

arXiv: https://arxiv.org/abs/1910.03197 · HTML: https://mlsanity.com/paper/1910.03197/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2019.

## Built on by (most cited first, 30 of 76)

- [FedCM: Federated Learning with Client-level Momentum](https://mlsanity.com/paper/2106.10874/) (2021)
- [Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization](https://mlsanity.com/paper/2310.15080/) (2023)
- [Decentralized Federated Learning: Balancing Communication and Computing Costs](https://mlsanity.com/paper/2107.12048/) (2021)
- [Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Models](https://mlsanity.com/paper/2305.11414/) (2023)
- [Gradient-Leakage Resilient Federated Learning](https://mlsanity.com/paper/2107.01154/) (2021)
- [FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Models](https://mlsanity.com/paper/2109.03775/) (2021)
- [L-DAWA: Layer-wise Divergence Aware Weight Aggregation in Federated Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/2307.07393/) (2023)
- [Implicit Gradient Alignment in Distributed and Federated Learning](https://mlsanity.com/paper/2106.13897/) (2021)
- [Cost-Effective Federated Learning in Mobile Edge Networks](https://mlsanity.com/paper/2109.05411/) (2021)
- [DELTA: Diverse Client Sampling for Fasting Federated Learning](https://mlsanity.com/paper/2205.13925/) (2022)
- [The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning](https://mlsanity.com/paper/2111.05326/) (2021)
- [Accelerated Federated Learning with Decoupled Adaptive Optimization](https://mlsanity.com/paper/2207.07223/) (2022)
- [AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks](https://mlsanity.com/paper/2303.00565/) (2023)
- [Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly](https://mlsanity.com/paper/2310.03150/) (2023)
- [A Decentralized Federated Learning Framework via Committee Mechanism with Convergence Guarantee](https://mlsanity.com/paper/2108.00365/) (2021)
- [Can Textual Gradient Work in Federated Learning?](https://mlsanity.com/paper/2502.19980/) (2025)
- [Semi-Synchronous Federated Learning for Energy-Efficient Training and Accelerated Convergence in Cross-Silo Settings](https://mlsanity.com/paper/2102.02849/) (2021)
- [FAIR-FATE: Fair Federated Learning with Momentum](https://mlsanity.com/paper/2209.13678/) (2022)
- [Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategies](https://mlsanity.com/paper/2109.03999/) (2021)
- [Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Data](https://mlsanity.com/paper/2203.01214/) (2022)
- [Federated Data Analytics: A Study on Linear Models](https://mlsanity.com/paper/2206.07786/) (2022)
- [Bridging the Gap Between Foundation Models and Heterogeneous Federated Learning](https://mlsanity.com/paper/2310.00247/) (2023)
- [From Deterioration to Acceleration: A Calibration Approach to Rehabilitating Step Asynchronism in Federated Optimization](https://mlsanity.com/paper/2112.09355/) (2021)
- [Federated Progressive Sparsification (Purge, Merge, Tune)+](https://mlsanity.com/paper/2204.12430/) (2022)
- [FedHyper: A Universal and Robust Learning Rate Scheduler for Federated Learning with Hypergradient Descent](https://mlsanity.com/paper/2310.03156/) (2023)
- [Confederated Learning: Federated Learning with Decentralized Edge Servers](https://mlsanity.com/paper/2205.14905/) (2022)
- [Advancements in Federated Learning: Models, Methods, and Privacy](https://mlsanity.com/paper/2302.11466/) (2023)
- [Gromov-Wasserstein Discrepancy with Local Differential Privacy for Distributed Structural Graphs](https://mlsanity.com/paper/2202.00808/) (2022)
- [Accelerating Asynchronous Federated Learning Convergence via Opportunistic Mobile Relaying](https://mlsanity.com/paper/2206.04742/) (2022)
- [Energy and Spectrum Efficient Federated Learning via High-Precision Over-the-Air Computation](https://mlsanity.com/paper/2208.07237/) (2022)

## Abstract (arXiv)

Federated learning (FL) provides a communication-efficient approach to solve machine learning problems concerning distributed data, without sending raw data to a central server. However, existing works on FL only utilize first-order gradient descent (GD) and do not consider the preceding iterations to gradient update which can potentially accelerate convergence. In this paper, we consider momentum term which relates to the last iteration. The proposed momentum federated learning (MFL) uses momentum gradient descent (MGD) in the local update step of FL system. We establish global convergence properties of MFL and derive an upper bound on MFL convergence rate. Comparing the upper bounds on MFL and FL convergence rate, we provide conditions in which MFL accelerates the convergence. For different machine learning models, the convergence performance of MFL is evaluated based on experiments with MNIST dataset. Simulation results comfirm that MFL is globally convergent and further reveal significant convergence improvement over FL.