mlsanityData to 7 Oct 2026

HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients

arXiv 2010.01264 ↗First posted 3 Oct 2020cs.LG stat.MLcited by 334

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a method of training machine learning models on private data distributed over a large number of possibly heterogeneous clients such as mobile phones and IoT devices. In this work, we propose a new federated learning framework named HeteroFL to address heterogeneous clients equipped with very different computation and communication capabilities. Our solution can enable the training of heterogeneous local models with varying computation complexities and still produce a single global inference model. For the first time, our method challenges the underlying assumption of existing work that local models have to share the same architecture as the global model. We demonstrate several strategies to enhance FL training and conduct extensive empirical evaluations, including five computation complexity levels of three model architecture on three datasets. We show that adaptively distributing subnetworks according to clients' capabilities is both computation and communication efficient.

Key facts. Cited by 334 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 32; 2022: 57; 2023: 77; 2024: 86; 2025: 52; 2026: 30.

334 papers.

Sort:

By area:

  1. 2021Towards Personalized Federated Learningcited by 462
  2. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
  3. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 184
  4. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  5. 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 140
  6. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
  7. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  8. 2024Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resourcescited by 93
  9. 2023SLoRA: Federated Parameter Efficient Fine-Tuning of Language Modelscited by 92
  10. 2021Federated Reconstruction: Partially Local Federated Learningcited by 78
  11. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  12. 2021Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Datacited by 69
38 more
  1. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  2. 2021Personalized Federated Learning through Local Memorizationcited by 57
  3. 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
  4. 2021Federated Learning for Internet of Things: Applications, Challenges, and Opportunitiescited by 49
  5. 2024FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learningcited by 47
  6. 2022FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clientscited by 46
  7. 2021SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Trainingcited by 45
  8. 2022pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learningcited by 44
  9. 2022A Survey on Heterogeneous Federated Learningcited by 42
  10. 2023Recent Advances on Federated Learning: A Systematic Surveycited by 40
  11. 2023Federated Learning for Computationally-Constrained Heterogeneous Devices: A Surveycited by 40
  12. 2022Test-Time Robust Personalization for Federated Learningcited by 39
  13. 2021Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspectivecited by 38
  14. 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
  15. 2023pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuningcited by 38
  16. 2023FedSN: A Federated Learning Framework over Heterogeneous LEO Satellite Networkscited by 38
  17. 2022No One Left Behind: Inclusive Federated Learning over Heterogeneous Devicescited by 36
  18. 2021Layer-wise Adaptive Model Aggregation for Scalable Federated Learningcited by 35
  19. 2023LOKI: Large-scale Data Reconstruction Attack against Federated Learning through Model Manipulationcited by 35
  20. 2022Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customizationcited by 34
  21. 2024Advances and Open Challenges in Federated Foundation Modelscited by 33
  22. 2021FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Modelscited by 31
  23. 2021FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorizationcited by 31
  24. 2023Towards Personalized Federated Learning via Heterogeneous Model Reassemblycited by 31
  25. 2023Why Batch Normalization Damage Federated Learning on Non-IID Data?cited by 29
  26. 2024A Survey on Efficient Federated Learning Methods for Foundation Model Trainingcited by 29
  27. 2022A Multi-agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learningcited by 28
  28. 2024Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boostingcited by 27
  29. 2023Federated Generalized Category Discoverycited by 26
  30. 2023Pruning Deep Neural Networks from a Sparsity Perspectivecited by 25
  31. 2022DELTA: Diverse Client Sampling for Fasting Federated Learningcited by 24
  32. 2023Every Parameter Matters: Ensuring the Convergence of Federated Learning with Dynamic Heterogeneous Models Reductioncited by 24
  33. 2021Towards Federated Learning on Time-Evolving Heterogeneous Datacited by 23
  34. 2022FedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networkscited by 23
  35. 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 23
  36. 2023A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiencycited by 21
  37. 2024pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learningcited by 21
  38. 2024Federated Model Heterogeneous Matryoshka Representation Learningcited by 21

50 most cited of 334 All 334 (JSON)