mlsanityData to 14 Oct 2026

Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysis

arXiv 2410.23131 ↗First posted 30 Oct 2024cs.LG cs.DCcited by 8

Not on the map. Why?

Abstract (arXiv)

In federated learning, it is common to assume that clients are always available to participate in training, which may not be feasible with user devices in practice. Recent works analyze federated learning under more realistic participation patterns, such as cyclic client availability or arbitrary participation. However, all such works either require strong assumptions (e.g., all clients participate almost surely within a bounded window), do not achieve linear speedup and reduced communication rounds, or are not applicable in the general non-convex setting. In this work, we focus on nonconvex optimization and consider participation patterns in which the chance of participation over a fixed window of rounds is equal among all clients, which includes cyclic client availability as a special case. Under this setting, we propose a new algorithm, named Amplified SCAFFOLD, and prove that it achieves linear speedup, reduced communication, and resilience to data heterogeneity simultaneously. In particular, for cyclic participation, our algorithm is proved to enjoy O(ε-2) communication rounds to find an ε-stationary point in the non-convex stochastic setting. In contrast, the prior work under the same setting requires O(kappa2 ε-4) communication rounds, where kappa denotes the data heterogeneity. Therefore, our algorithm significantly reduces communication rounds due to better dependency in terms of ε and kappa. Our analysis relies on a fine-grained treatment of the nested dependence between client participation and errors in the control variates, which results in tighter guarantees than previous work. We also provide experimental results with (1) synthetic data and (2) real-world data with a large number of clients (N = 250), demonstrating the effectiveness of our algorithm under periodic client participation.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 34 of its 53 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

34 of 53 references.

References · 34

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  6. 2020Adaptive Federated Optimizationcited by 1,046
  7. 2019Towards Federated Learning at Scale: System Designcited by 832
  8. 2018Local SGD Converges Fast and Communicates Littlecited by 605
  9. 2021A Field Guide to Federated Optimizationcited by 315
  10. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 204
  11. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 198
  12. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
22 more
  1. 2017Asynchronous Decentralized Parallel Stochastic Gradient Descentcited by 159
  2. 2017Collecting Telemetry Data Privatelycited by 149
  3. 2020Optimal Client Sampling for Federated Learningcited by 124
  4. 2018Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithmscited by 106
  5. 2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
  6. 2021Papaya: Practical, Private, and Scalable Federated Learningcited by 90
  7. 2020Federated Accelerated Stochastic Gradient Descentcited by 85
  8. 2021Federated Evaluation and Tuning for On-Device Personalization: System Design & Applicationscited by 85
  9. 2021Fast Federated Learning in the Presence of Arbitrary Device Unavailabilitycited by 72
  10. 2020Towards Flexible Device Participation in Federated Learningcited by 65
  11. 2022A Unified Analysis of Federated Learning with Arbitrary Client Participationcited by 45
  12. 2021Anarchic Federated Learningcited by 42
  13. 2019Semi-Cyclic Stochastic Gradient Descentcited by 40
  14. 2021Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspectivecited by 38
  15. 2023Momentum Benefits Non-IID Federated Learning Simply and Provablycited by 38
  16. 2017Learning Differentially Private Recurrent Language Modelscited by 36
  17. 2023On the Convergence of Federated Averaging with Cyclic Client Participationcited by 31
  18. 2020Distributed Non-Convex Optimization with Sublinear Speedup under Intermittent Client Availabilitycited by 20
  19. 2023Federated Learning with Regularized Client Participationcited by 14
  20. 2022Sharper Rates and Flexible Framework for Nonconvex SGD with Client and Data Samplingcited by 10
  21. 2023Improving Accelerated Federated Learning with Compression and Importance Samplingcited by 8
  22. 2023A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averagingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 7.

8 papers.

  1. 2026GFPL: Generative Federated Prototype Learning for Resource-Constrained and Data-Imbalanced Vision Taskcited by 1
  2. 2025Computation-aware Energy-harvesting Federated Learning with Pipelined Cyclic Schedulingcited by 0
  3. 2026Communication-Efficient Federated AUC Maximization with Cyclic Client Participationcited by 0
  4. 2026Ordered Local Momentum for Asynchronous Distributed Learning under Arbitrary Delayscited by 0
  5. 2026RefProtoFL: Communication-Efficient Federated Learning via External-Referenced Prototype Alignmentcited by 0
  6. 2026Federated Learning of Binary Neural Networks: Enabling Low-Cost Inferencecited by 0
  7. 2026Agentic Federated Learning: The Future of Distributed Training Orchestrationcited by 0
  8. 2026Over-the-Air Federated Learning in Heterogeneous Mobile Wireless Networkscited by 0