mlsanityData to 7 Oct 2026

A Convergence Theory for Federated Average: Beyond Smoothness

arXiv 2211.01588 ↗First posted 3 Nov 2022cs.LG stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

Federated learning enables a large amount of edge computing devices to learn a model without data sharing jointly. As a leading algorithm in this setting, Federated Average FedAvg, which runs Stochastic Gradient Descent (SGD) in parallel on local devices and averages the sequences only once in a while, have been widely used due to their simplicity and low communication cost. However, despite recent research efforts, it lacks theoretical analysis under assumptions beyond smoothness. In this paper, we analyze the convergence of FedAvg. Different from the existing work, we relax the assumption of strong smoothness. More specifically, we assume the semi-smoothness and semi-Lipschitz properties for the loss function, which have an additional first-order term in assumption definitions. In addition, we also assume bound on the gradient, which is weaker than the commonly used bounded gradient assumption in the convergence analysis scheme. As a solution, this paper provides a theoretical convergence study on Federated Learning.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 30 of its 37 references that are in the map. First posted on arXiv 3 Nov 2022.

Builds on

30 of 37 references.

References · 30

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  6. 2018Federated Learning with Non-IID Datacited by 1,099
  7. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  8. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  9. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  10. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  11. 2020Federated Learning with Matched Averagingcited by 564
  12. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
18 more
  1. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  2. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
  3. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 306
  4. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  5. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  6. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 202
  7. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 179
  8. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  9. 2017Convergence Analysis of Two-layer Neural Networks with ReLU Activationcited by 137
  10. 2020Multi-site fMRI Analysis Using Privacy-preserving Federated Learning and Domain Adaptation: ABIDE Resultscited by 122
  11. 2019Federated Learning for Emoji Prediction in a Mobile Keyboardcited by 115
  12. 2020Local SGD: Unified Theory and New Efficient Methodscited by 88
  13. 2019First Analysis of Local GD on Heterogeneous Datacited by 80
  14. 2019Federated Learning Of Out-Of-Vocabulary Wordscited by 51
  15. 2022Sketching for First Order Method: Efficient Algorithm for Low-Bandwidth Channel and Vulnerabilitycited by 25
  16. 2021On Convergence of Federated Averaging Langevin Dynamicscited by 14
  17. 2019Federated Learning with Additional Mechanisms on Clients to Reduce Communication Costscited by 12
  18. 2019Communication trade-offs for synchronized distributed SGD with large step sizecited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 1; 2026: 2.

4 papers.

  1. 2023An effective theory of collective deep learningcited by 7
  2. 2025Noise Resilient Over-The-Air Federated Learning In Heterogeneous Wireless Networkscited by 1
  3. 2026ERIS: Enhancing Privacy and Scalability in Federated Learning via Federated Shard Aggregationcited by 0
  4. 2026FL-Sailer: Efficient and Privacy-Preserving Federated Learning for Scalable Single-Cell Epigenetic Data Analysis via Adaptive Samplingcited by 0