mlsanityData to 14 Oct 2026

Embracing Federated Learning: Enabling Weak Client Participation via Partial Model Training

arXiv 2406.15125 ↗First posted 21 Jun 2024cs.LG cs.DCcited by 3

Not on the map. Why?

Abstract (arXiv)

In Federated Learning (FL), clients may have weak devices that cannot train the full model or even hold it in their memory space. To implement large-scale FL applications, thus, it is crucial to develop a distributed learning method that enables the participation of such weak clients. We propose EmbracingFL, a general FL framework that allows all available clients to join the distributed training regardless of their system resource capacity. The framework is built upon a novel form of partial model training method in which each client trains as many consecutive output-side layers as its system resources allow. Our study demonstrates that EmbracingFL encourages each layer to have similar data representations across clients, improving FL efficiency. The proposed partial model training method guarantees convergence to a neighbor of stationary points for non-convex and smooth problems. We evaluate the efficacy of EmbracingFL under a variety of settings with a mixed number of strong, moderate (~40% memory), and weak (~15% memory) clients, datasets (CIFAR-10, FEMNIST, and IMDB), and models (ResNet20, CNN, and LSTM). Our empirical study shows that EmbracingFL consistently achieves high accuracy as like all clients are strong, outperforming the state-of-the-art width reduction methods (i.e. HeteroFL and FjORD).

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 30 of its 57 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

30 of 57 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  4. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  5. 2018LEAF: A Benchmark for Federated Settingscited by 792
  6. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  7. 2020Federated Learning with Matched Averagingcited by 564
  8. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
  9. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  10. 2020SplitFed: When Federated Learning Meets Split Learningcited by 310
  11. 2018Distributed learning of deep neural network over multiple agentscited by 279
  12. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
18 more
  1. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  2. 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 141
  3. 2018Greedy Layerwise Learning Can Scale to ImageNetcited by 115
  4. 2022Layer-wised Model Aggregation for Personalized Federated Learningcited by 96
  5. 2020MNN: A Universal and Efficient Inference Enginecited by 65
  6. 2021Federated Learning for Internet of Things: Applications, Challenges, and Opportunitiescited by 50
  7. 2022No One Left Behind: Inclusive Federated Learning over Heterogeneous Devicescited by 37
  8. 2022Automated Progressive Learning for Efficient Training of Vision Transformerscited by 37
  9. 2021Layer-wise Adaptive Model Aggregation for Scalable Federated Learningcited by 36
  10. 2021FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorizationcited by 31
  11. 2021ResIST: Layer-Wise Decomposition of ResNets for Distributed Trainingcited by 18
  12. 2021Efficient and Private Federated Learning with Partially Trainable Networkscited by 13
  13. 2021FedHe: Heterogeneous Models and Communication-Efficient Federated Learningcited by 13
  14. 2021FedPrune: Towards Inclusive Federated Learningcited by 12
  15. 2020A Closer Look at Codistillation for Distributed Trainingcited by 8
  16. 2022Partial Model Averaging in Federated Learning: Performance Guarantees and Benefitscited by 8
  17. 2023Efficient Self-supervised Continual Learning with Progressive Task-correlated Layer Freezingcited by 6
  18. 2021Progressive Stage-wise Learning for Unsupervised Feature Representation Enhancementcited by 3

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Asynchronous Sharpness-Aware Minimization For Fast and Accurate Deep Learningcited by 2
  2. 2025Enabling Weak Client Participation via On-device Knowledge Distillation in Heterogeneous Federated Learningcited by 1
  3. 2025Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Reviewcited by 0