mlsanityData to 7 Oct 2026

On the Convergence of Federated Averaging with Cyclic Client Participation

arXiv 2302.03109 ↗First posted 6 Feb 2023cs.LG cs.DCcited by 31

Not on the map. Why?

Abstract (arXiv)

Federated Averaging (FedAvg) and its variants are the most popular optimization algorithms in federated learning (FL). Previous convergence analyses of FedAvg either assume full client participation or partial client participation where the clients can be uniformly sampled. However, in practical cross-device FL systems, only a subset of clients that satisfy local criteria such as battery status, network connectivity, and maximum participation frequency requirements (to ensure privacy) are available for training at a given time. As a result, client availability follows a natural cyclic pattern. We provide (to our knowledge) the first theoretical framework to analyze the convergence of FedAvg with cyclic client participation with several different client optimizers such as GD, SGD, and shuffled SGD. Our analysis discovers that cyclic client participation can achieve a faster asymptotic convergence rate than vanilla FedAvg with uniform client participation under suitable conditions, providing valuable insights into the design of client sampling protocols.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 54 of its 60 references that are in the map. First posted on arXiv 6 Feb 2023.

Builds on

54 of 60 references.

References · 54

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  6. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  7. 2020Adaptive Federated Optimizationcited by 1,036
  8. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  9. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  10. 2019Towards Federated Learning at Scale: System Designcited by 826
  11. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  12. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
38 more
  1. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
  2. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
  3. 2021A Field Guide to Federated Optimizationcited by 312
  4. 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 231
  5. 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
  6. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 202
  7. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 196
  8. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  9. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 179
  10. 2016Practical Secure Aggregation for Federated Learning on User-Held Datacited by 178
  11. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 178
  12. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 174
  13. 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 169
  14. 2020Is Local SGD Better than Minibatch SGD?cited by 151
  15. 2019On the Convergence of Local Descent Methods in Federated Learningcited by 150
  16. 2020Minibatch vs Local SGD for Heterogeneous Distributed Learningcited by 147
  17. 2019SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcited by 121
  18. 2020Random Reshuffling: Simple Analysis with Vast Improvementscited by 112
  19. 2019Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronizationcited by 106
  20. 2021Papaya: Practical, Private, and Scalable Federated Learningcited by 89
  21. 2021Federated Evaluation and Tuning for On-Device Personalization: System Design & Applicationscited by 84
  22. 2015Why Random Reshuffling Beats Stochastic Gradient Descentcited by 78
  23. 2020SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolationcited by 66
  24. 2020Towards Flexible Device Participation in Federated Learningcited by 64
  25. 2019How Good is SGD with Random Shuffling?cited by 63
  26. 2019Active Federated Learningcited by 63
  27. 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 62
  28. 2019SGD without Replacement: Sharper Rates for General Smooth Convex Functionscited by 59
  29. 2020A Unified Convergence Analysis for Shuffling-Type Gradient Methodscited by 57
  30. 2020SGD with shuffling: optimal rates without component convexity and large epoch requirementscited by 57
  31. 2020Closing the convergence gap of SGD without replacementcited by 56
  32. 2022Multi-Epoch Matrix Factorization Mechanisms for Private Machine Learningcited by 48
  33. 2022A Unified Analysis of Federated Learning with Arbitrary Client Participationcited by 45
  34. 2019Semi-Cyclic Stochastic Gradient Descentcited by 40
  35. 2020Privacy Amplification via Random Check-Inscited by 39
  36. 2022On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Datacited by 36
  37. 2021Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyondcited by 35
  38. 2015Convergence Rate of Incremental Gradient and Newton Methodscited by 26

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 15; 2025: 6; 2026: 2.

31 papers.

Sort:

  1. 2023Federated Learning of Gboard Language Models with Differential Privacycited by 64
  2. 2023AsGrad: A Sharp Unified Analysis of Asynchronous-SGD Algorithmscited by 16
  3. 2024Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptationcited by 12
  4. 2024Efficient Federated Learning against Heterogeneous and Non-stationary Client Unavailabilitycited by 9
  5. 2024One-Shot Sequential Federated Learning for Non-IID Data by Enhancing Local Model Diversitycited by 8
  6. 2024Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysiscited by 8
  7. 2023On Convergence of Incremental Gradient for Non-Convex Smooth Functionscited by 7
  8. 2023Improving Accelerated Federated Learning with Compression and Importance Samplingcited by 7
  9. 2023Distributed Personalized Empirical Risk Minimizationcited by 6
  10. 2024FedStale: leveraging stale client updates in federated learningcited by 5
  11. 2024DRACO: Decentralized Asynchronous Federated Learning over Row-Stochastic Wireless Networkscited by 5
  12. 2024Save It All: Enabling Full Parameter Tuning for Federated Large Language Models via Cycle Block Gradient Descentcited by 4
19 more
  1. 2023Communication-Efficient Heterogeneous Federated Learning with Generalized Heavy-Ball Momentumcited by 3
  2. 2024Empowering Federated Learning with Implicit Gossiping: Mitigating Connection Unreliability Amidst Unknown and Arbitrary Dynamicscited by 3
  3. 2024Understanding Server-Assisted Federated Learning in the Presence of Incomplete Client Participationcited by 3
  4. 2024Unity is Power: Semi-Asynchronous Collaborative Training of Large-Scale Models with Structured Pruning in Resource-Limited Clientscited by 3
  5. 2024Federated Learning Clients Clustering with Adaptation to Data Driftscited by 3
  6. 2024Asynchronous Federated Learning: A Scalable Approach for Decentralized Machine Learningcited by 3
  7. 2024SCALA: Split Federated Learning with Concatenated Activations and Logit Adjustmentscited by 2
  8. 2025Battery-aware Cyclic Scheduling in Energy-harvesting Federated Learningcited by 2
  9. 2023A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averagingcited by 1
  10. 2023Enhanced Federated Optimization: Adaptive Unbiased Client Sampling with Reduced Variancecited by 1
  11. 2024Debiasing Federated Learning with Correlated Client Participationcited by 1
  12. 2025On the Convergence and Stability of Distributed Sub-model Trainingcited by 1
  13. 2024Client2Vec: Improving Federated Learning by Distribution Shifts Aware Client Indexingcited by 0
  14. 2025Analysis of Asynchronous Federated Learning: Unraveling the Interactions between Gradient Compression, Delay, and Data Heterogeneitycited by 0
  15. 2025Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Reviewcited by 0
  16. 2025Incentivizing High-quality Participation From Federated Learning Agentscited by 0
  17. 2025Computation-aware Energy-harvesting Federated Learning with Pipelined Cyclic Schedulingcited by 0
  18. 2026Communication-Efficient Federated AUC Maximization with Cyclic Client Participationcited by 0
  19. 2026From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learningcited by 0