mlsanityData to 7 Oct 2026

Accelerating Hybrid Federated Learning Convergence under Partial Participation

arXiv 2304.05397 ↗First posted 10 Apr 2023cs.DC cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Over the past few years, Federated Learning (FL) has become a popular distributed machine learning paradigm. FL involves a group of clients with decentralized data who collaborate to learn a common model under the coordination of a centralized server, with the goal of protecting clients' privacy by ensuring that local datasets never leave the clients and that the server only performs model aggregation. However, in realistic scenarios, the server may be able to collect a small amount of data that approximately mimics the population distribution and has stronger computational ability to perform the learning process. To address this, we focus on the hybrid FL framework in this paper. While previous hybrid FL work has shown that the alternative training of clients and server can increase convergence speed, it has focused on the scenario where clients fully participate and ignores the negative effect of partial participation. In this paper, we provide theoretical analysis of hybrid FL under clients' partial participation to validate that partial participation is the key constraint on convergence speed. We then propose a new algorithm called FedCLG, which investigates the two-fold role of the server in hybrid FL. Firstly, the server needs to process the training steps using its small amount of local datasets. Secondly, the server's calculated gradient needs to guide the participated clients' training and the server's aggregation. We validate our theoretical findings through numerical experiments, which show that our proposed method FedCLG outperforms state-of-the-art methods.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 29 of its 50 references that are in the map. First posted on arXiv 10 Apr 2023.

Builds on

29 of 50 references.

References · 29

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  3. 2018Federated Learning with Non-IID Datacited by 1,099
  4. 2019Towards Federated Learning at Scale: System Designcited by 826
  5. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  6. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  7. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  8. 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 401
  9. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  10. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  11. 2020FLTrust: Byzantine-robust Federated Learning via Trust Bootstrappingcited by 330
  12. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
17 more
  1. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  2. 2019A Survey on Distributed Machine Learningcited by 187
  3. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 178
  4. 2016Stochastic Variance Reduction for Nonconvex Optimizationcited by 143
  5. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  6. 2020Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learningcited by 112
  7. 2020UVeQFed: Universal Vector Quantization for Federated Learningcited by 103
  8. 2021Fast Federated Learning in the Presence of Arbitrary Device Unavailabilitycited by 72
  9. 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 62
  10. 2021Adaptive Quantization of Model Updates for Communication-Efficient Federated Learningcited by 60
  11. 2020Deep Anomaly Detection for Time-series Data in Industrial IoT: A Communication-Efficient On-device Federated Learning Approachcited by 56
  12. 2019HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learningcited by 51
  13. 2022A Unified Analysis of Federated Learning with Arbitrary Client Participationcited by 45
  14. 2016Parallel SGD: When does averaging help?cited by 30
  15. 2021A Hybrid Architecture for Federated and Centralized Learningcited by 8
  16. 2022Mixed Federated Learning: Joint Decentralized and Centralized Learningcited by 8
  17. 2021To Talk or to Work: Delay Efficient Federated Learning over Mobile Edge Devicescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 8; 2026: 2.

15 papers.

Sort:

By area:

  1. 2025FedAdamW: A Communication-Efficient Optimizer with Convergence and Generalization Guarantees for Federated Large Modelscited by 15
  2. 2025FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-World LoRAcited by 13
  3. 2025Federated Learning for Diffusion Modelscited by 6
  4. 2025Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuningcited by 5
  5. 2024Generalization Error Matters in Decentralized Learning Under Byzantine Attackscited by 4
  6. 2025FedCanon: Non-Convex Composite Federated Learning with Efficient Proximal Operation on Heterogeneous Datacited by 4
  7. 2025FedEL: Federated Elastic Learning for Heterogeneous Devicescited by 3
  8. 2025Mitigating Modality Quantity and Quality Imbalance in Multimodal Online Federated Learningcited by 2
  9. 2026FedTreeLoRA: Reconciling Statistical and Functional Heterogeneity in Federated LoRA Fine-Tuningcited by 2
  10. 2024Rethinking the Starting Point: Collaborative Pre-Training for Federated Downstream Taskscited by 1
  11. 2024Emulating Full Participation: An Effective and Fair Client Selection Strategy for Federated Learningcited by 1
  12. 2024Debiasing Federated Learning with Correlated Client Participationcited by 1
3 more
  1. 2025Generalization Error Analysis for Attack-Free and Byzantine-Resilient Decentralized Learning with Data Heterogeneitycited by 1
  2. 2023On the Convergence of Federated Averaging under Partial Participation for Over-parameterized Neural Networkscited by 0
  3. 2026From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learningcited by 0