mlsanityData to 14 Oct 2026

Anchor Sampling for Federated Learning with Partial Client Participation

arXiv 2206.05891 ↗First posted 13 Jun 2022cs.LG cs.DC stat.MLcited by 14

Not on the map. Why?

Abstract (arXiv)

Compared with full client participation, partial client participation is a more practical scenario in federated learning, but it may amplify some challenges in federated learning, such as data heterogeneity. The lack of inactive clients' updates in partial client participation makes it more likely for the model aggregation to deviate from the aggregation based on full client participation. Training with large batches on individual clients is proposed to address data heterogeneity in general, but their effectiveness under partial client participation is not clear. Motivated by these challenges, we propose to develop a novel federated learning framework, referred to as FedAMD, for partial client participation. The core idea is anchor sampling, which separates partial participants into anchor and miner groups. Each client in the anchor group aims at the local bullseye with the gradient computation using a large batch. Guided by the bullseyes, clients in the miner group steer multiple near-optimal local updates using small batches and update the global model. By integrating the results of the two groups, FedAMD is able to accelerate the training process and improve the model performance. Measured by ε-approximation and compared to the state-of-the-art methods, FedAMD achieves the convergence by up to O(1/ε) fewer communication rounds under non-convex objectives. Empirical studies on real-world datasets validate the effectiveness of FedAMD and demonstrate the superiority of the proposed algorithm: Not only does it considerably save computation and communication costs, but also the test accuracy significantly improves.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 65 of its 91 references that are in the map. First posted on arXiv 13 Jun 2022.

Builds on

65 of 91 references.

References · 65

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  6. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  7. 2018Federated Learning with Non-IID Datacited by 1,108
  8. 2016An overview of gradient descent optimization algorithmscited by 1,033
  9. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  10. 2018Local SGD Converges Fast and Communicates Littlecited by 605
  11. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 539
  12. 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 402
38 more
  1. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  2. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 278
  3. 2017SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradientcited by 220
  4. 2021Personalized Federated Learning using Hypernetworkscited by 212
  5. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 202
  6. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 195
  7. 2018Measuring the Effects of Data Parallelism on Neural Network Trainingcited by 193
  8. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  9. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 180
  10. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 179
  11. 2020Better Theory for SGD in the Nonconvex Worldcited by 158
  12. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 154
  13. 2019On the Convergence of Local Descent Methods in Federated Learningcited by 153
  14. 2020Is Local SGD Better than Minibatch SGD?cited by 151
  15. 2020Minibatch vs Local SGD for Heterogeneous Distributed Learningcited by 147
  16. 2016Stochastic Variance Reduction for Nonconvex Optimizationcited by 145
  17. 2013Mini-batch Stochastic Approximation Methods for Nonconvex Stochastic Composite Optimizationcited by 143
  18. 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 123
  19. 2020PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimizationcited by 119
  20. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 109
  21. 2020FedSplit: An algorithmic framework for fast federated optimizationcited by 108
  22. 2019Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronizationcited by 107
  23. 2020Federated Accelerated Stochastic Gradient Descentcited by 85
  24. 2019Variance Reduced Local SGD with Lower Communication Complexitycited by 80
  25. 2021MARINA: Faster Non-Convex Distributed Learning with Compressioncited by 78
  26. 2020From Local SGD to Local Fixed-Point Methods for Federated Learningcited by 74
  27. 2016Variance Reduction for Faster Non-Convex Optimizationcited by 72
  28. 2021Fast Federated Learning in the Presence of Arbitrary Device Unavailabilitycited by 72
  29. 2020Towards Flexible Device Participation in Federated Learningcited by 65
  30. 2016Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifoldscited by 64
  31. 2018Stochastic Nested Variance Reduction for Nonconvex Optimizationcited by 63
  32. 2017Non-convex Finite-Sum Optimization Via SCSG Methodscited by 61
  33. 2020Bidirectional compression in heterogeneous settings for distributed or federated learning with partial participation: tight convergence guaranteescited by 46
  34. 2019Semi-Cyclic Stochastic Gradient Descentcited by 40
  35. 2021One for One, or All for All: Equilibria and Optimality of Collaboration in Federated Learningcited by 39
  36. 2021Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyondcited by 36
  37. 2015An optimal randomized incremental gradient methodcited by 33
  38. 2021Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learningcited by 31

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 7; 2026: 2.

14 papers.

Sort:

By area:

  1. 2024FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Modelcited by 74
  2. 2024FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extractioncited by 21
  3. 2024Towards Federated RLHF with Aggregated Client Preference for LLMscited by 13
  4. 2024Hierarchical Federated Learning with Multi-Timescale Gradient Correctioncited by 7
  5. 2025Scalable Decentralized Learning with Teleportationcited by 3
  6. 2024Efficient Federated Learning against Byzantine Attacks and Data Heterogeneity via Aggregating Normalized Gradientscited by 2
  7. 2025FedFetch: Faster Federated Learning with Adaptive Downstream Prefetchingcited by 2
  8. 2025Accelerated Training of Federated Learning via Second-Order Methodscited by 0
  9. 2025Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Reviewcited by 0
  10. 2025BadPromptFL: A Novel Backdoor Threat to Prompt-based Federated Learning in Multimodal Modelscited by 0
  11. 2025Towards Privacy-Preserving and Heterogeneity-aware Split Federated Learning via Probabilistic Maskingcited by 0
  12. 2025FedDPC : Handling Data Heterogeneity and Partial Client Participation in Federated Learningcited by 0
2 more
  1. 2026Dimensionality Reduction for Robust Federated Learning: A Theoretical Analysis and Convergence Guaranteecited by 0
  2. 2026From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learningcited by 0