mlsanityData to 14 Oct 2026

Federated Learning with Regularized Client Participation

arXiv 2302.03662 ↗First posted 7 Feb 2023cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a distributed machine learning approach where multiple clients work together to solve a machine learning task. One of the key challenges in FL is the issue of partial participation, which occurs when a large number of clients are involved in the training process. The traditional method to address this problem is randomly selecting a subset of clients at each communication round. In our research, we propose a new technique and design a novel regularized client participation scheme. Under this scheme, each client joins the learning process every R communication rounds, which we refer to as a meta epoch. We have found that this participation scheme leads to a reduction in the variance caused by client sampling. Combined with the popular FedAvg algorithm (McMahan et al., 2017), it results in superior rates under standard assumptions. For instance, the optimization term in our main convergence bound decreases linearly with the product of the number of communication rounds and the size of the local dataset of each client, and the statistical term scales with step size quadratically instead of linearly (the case for client sampling with replacement), leading to better convergence rate Oleft(frac1T2right) compared to Oleft(frac1Tright), where T is the total number of communication rounds. Furthermore, our results permit arbitrary client availability as long as each client is available for training once per each meta epoch.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 54 of its 71 references that are in the map. First posted on arXiv 7 Feb 2023.

Builds on

54 of 71 references.

References · 54

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2020The Future of Digital Health with Federated Learningcited by 633
  3. 2018Local SGD Converges Fast and Communicates Littlecited by 605
  4. 2018Federated Learning for Mobile Keyboard Predictioncited by 572
  5. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 408
  6. 2018Sparsified SGD with Memorycited by 376
  7. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 367
  8. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  9. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 278
  10. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  11. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 256
  12. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 238
38 more
  1. 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
  2. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 202
  3. 2017Gradient Sparsification for Communication-Efficient Distributed Optimizationcited by 201
  4. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 198
  5. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  6. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  7. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 175
  8. 2016QSGD: Communication-Efficient SGD via Gradient Quantization and Encodingcited by 147
  9. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  10. 2018Protection Against Reconstruction and Its Applications in Private Federated Learningcited by 130
  11. 2020Optimal Client Sampling for Federated Learningcited by 124
  12. 2020Random Reshuffling: Simple Analysis with Vast Improvementscited by 115
  13. 2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
  14. 2019Natural Compression for Distributed Deep Learningcited by 90
  15. 2020On the Convergence of SGD with Biased Gradientscited by 83
  16. 2015Why Random Reshuffling Beats Stochastic Gradient Descentcited by 79
  17. 2020From Local SGD to Local Fixed-Point Methods for Federated Learningcited by 74
  18. 2021Fast Federated Learning in the Presence of Arbitrary Device Unavailabilitycited by 72
  19. 2021Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Samplingcited by 66
  20. 2020Towards Flexible Device Participation in Federated Learningcited by 65
  21. 2020Fast-Convergent Federated Learningcited by 61
  22. 2015Communication Complexity of Distributed Convex Learning and Optimizationcited by 60
  23. 2020SGD with shuffling: optimal rates without component convexity and large epoch requirementscited by 58
  24. 2022A Unified Analysis of Federated Learning with Arbitrary Client Participationcited by 45
  25. 2020A Better Alternative to Error Feedback for Communication-Efficient Distributed Learningcited by 43
  26. 2021Anarchic Federated Learningcited by 42
  27. 2016Randomized Distributed Mean Estimation: Accuracy vs Communicationcited by 39
  28. 2020Communication-Efficient Federated Learning via Optimal Client Samplingcited by 39
  29. 2021Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyondcited by 36
  30. 2021Proximal and Federated Random Reshufflingcited by 33
  31. 2022Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learningcited by 33
  32. 2022Server-Side Stepsizes and Sampling Without Replacement Provably Help in Federated Optimizationcited by 23
  33. 2022Can 5th Generation Local Training Methods Support Client Sampling? Yes!cited by 22
  34. 2022FedShuffle: Recipes for Better Use of Local Work in Federated Learningcited by 21
  35. 2020Distributed Non-Convex Optimization with Sublinear Speedup under Intermittent Client Availabilitycited by 20
  36. 2021Random Reshuffling with Variance Reduction: New Analysis and Better Ratescited by 19
  37. 2022Federated Optimization Algorithms with Random Reshuffling and Gradient Compressioncited by 18
  38. 2022Federated Random Reshuffling with Compression and Variance Reductioncited by 12

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 5; 2025: 3; 2026: 1.

14 papers.

Sort:

By area:

  1. 2023Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Uglycited by 17
  2. 2024Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptationcited by 13
  3. 2025Federated Continual Learning: Concepts, Challenges, and Solutionscited by 13
  4. 2023Improving Accelerated Federated Learning with Compression and Importance Samplingcited by 8
  5. 2024Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysiscited by 8
  6. 2023Towards a Better Theoretical Understanding of Independent Subnetwork Trainingcited by 6
  7. 2024Federated Learning and RAG Integration: A Scalable Approach for Medical Large Language Modelscited by 4
  8. 2024Understanding Server-Assisted Federated Learning in the Presence of Incomplete Client Participationcited by 3
  9. 2023Enhanced Federated Optimization: Adaptive Unbiased Client Sampling with Reduced Variancecited by 2
  10. 2023A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averagingcited by 1
  11. 2024Federated Learning and AI Regulation in the European Union: Who is Responsible? -- An Interdisciplinary Analysiscited by 1
  12. 2025First Provable Guarantees for Practical Private FL: Beyond Restrictive Assumptionscited by 1
2 more
  1. 2025Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Reviewcited by 0
  2. 2026Theoretical Foundations of Communication-Efficient, Robust, and Practical Distributed and Federated Optimizationcited by 0