mlsanityData to 7 Oct 2026

Federated Stochastic Gradient Descent Begets Self-Induced Momentum

arXiv 2202.08402 ↗First posted 17 Feb 2022cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is an emerging machine learning method that can be applied in mobile edge systems, in which a server and a host of clients collaboratively train a statistical model utilizing the data and computation resources of the clients without directly exposing their privacy-sensitive data. We show that running stochastic gradient descent (SGD) in such a setting can be viewed as adding a momentum-like term to the global aggregation process. Based on this finding, we further analyze the convergence rate of a federated learning system by accounting for the effects of parameter staleness and communication resources. These results advance the understanding of the Federated SGD algorithm, and also forges a link between staleness analysis and federated computing systems, which can be useful for systems designers.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 22 references that are in the map. First posted on arXiv 17 Feb 2022.

Builds on

20 of 22 references.

References · 20

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  3. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  4. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  5. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  6. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
  7. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  8. 2019A Joint Learning and Communications Framework for Federated Learning over Wireless Networkscited by 262
  9. 2018Federated Learning via Over-the-Air Computationcited by 215
  10. 2019The Roadmap to 6G -- AI Empowered Wireless Networkscited by 150
  11. 2019Scheduling Policies for Federated Learning in Wireless Networkscited by 131
  12. 2018LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learningcited by 95
8 more
  1. 2018Wireless Network Intelligence at the Edgecited by 90
  2. 2019Accelerating Federated Learning via Momentum Gradient Descentcited by 75
  3. 2018A Tight Convergence Analysis for Stochastic Gradient Descent with Delayed Updatescited by 47
  4. 2017Communication-Efficient Algorithms for Decentralized and Stochastic Optimizationcited by 42
  5. 2019Age-Based Scheduling Policy for Federated Learning in Mobile Edge Networkscited by 39
  6. 2016Asynchrony begets Momentum, with an Application to Deep Learningcited by 30
  7. 2018Toward Understanding the Impact of Staleness in Distributed Machine Learningcited by 28
  8. 2020Faster On-Device Training Using New Federated Momentum Algorithmcited by 25

Built on by

Not cited yet.