mlsanityData to 14 Oct 2026

Heterogeneous Federated Learning: State-of-the-art and Research Challenges

arXiv 2307.10616 ↗First posted 20 Jul 2023cs.LG cs.AI cs.CVcited by 161

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) has drawn increasing attention owing to its potential use in large-scale industrial applications. Existing federated learning works mainly focus on model homogeneous settings. However, practical federated learning typically faces the heterogeneity of data distributions, model architectures, network environments, and hardware devices among participant clients. Heterogeneous Federated Learning (HFL) is much more challenging, and corresponding solutions are diverse and complex. Therefore, a systematic survey on this topic about the research challenges and state-of-the-art is essential. In this survey, we firstly summarize the various research challenges in HFL from five aspects: statistical heterogeneity, model heterogeneity, communication heterogeneity, device heterogeneity, and additional challenges. In addition, recent advances in HFL are reviewed and a new taxonomy of existing HFL methods is proposed with an in-depth analysis of their pros and cons. We classify existing methods from three different levels according to the HFL procedure: data-level, model-level, and server-level. Finally, several critical and promising future research directions in HFL are discussed, which may facilitate further developments in this field. A periodically updated collection on HFL is available at https://github.com/marswhu/HFL_Survey.

Key facts. Cited by 161 papers in the map (data through 14 Oct 2026). Builds on 182 of its 264 references that are in the map. First posted on arXiv 20 Jul 2023.

Builds on

182 of 264 references.

References · 182

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  5. 2019Advances and Open Problems in Federated Learningcited by 3,046
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  7. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  8. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  9. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  10. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  11. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  12. 2019Towards Federated Learning at Scale: System Designcited by 832
38 more
  1. 2018LEAF: A Benchmark for Federated Settingscited by 792
  2. 2021Model-Contrastive Federated Learningcited by 766
  3. 2018How To Backdoor Federated Learningcited by 765
  4. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  5. 2017Federated Multi-Task Learningcited by 672
  6. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  7. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  8. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  9. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  10. 2019Federated Learning with Personalization Layerscited by 541
  11. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  12. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  13. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  14. 2021Towards Personalized Federated Learningcited by 464
  15. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  16. 2019Agnostic Federated Learningcited by 456
  17. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
  18. 2019Federated Machine Learning: Concept and Applicationscited by 412
  19. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 408
  20. 2019Fair Resource Allocation in Federated Learningcited by 394
  21. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  22. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
  23. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 352
  24. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  25. 2021Federated Learning on Non-IID Data: A Surveycited by 346
  26. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  27. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  28. 2020Attack of the Tails: Yes, You Really Can Backdoor Federated Learningcited by 309
  29. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 300
  30. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 283
  31. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 275
  32. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  33. 2019Can You Really Backdoor Federated Learning?cited by 255
  34. 2021Federated Learning for Internet of Things: A Comprehensive Surveycited by 252
  35. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 243
  36. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  37. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 223
  38. 2020Federated Learning of a Mixture of Global and Local Modelscited by 210

50 most cited of 182 All 182 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 57; 2025: 63; 2026: 34.

161 papers.

Sort:

By area:

  1. 2024A Survey on LoRA of Large Language Modelscited by 67
  2. 2023Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learningcited by 40
  3. 2024Fair Federated Learning under Domain Skew with Local Consistency and Domain Diversitycited by 30
  4. 2023Think Twice Before Selection: Federated Evidential Active Learning for Medical Image Analysis with Domain Shiftscited by 28
  5. 2024Unleashing the Power of Continual Learning on Non-Centralized Devices: A Surveycited by 23
  6. 2024Vertical Federated Learning for Effectiveness, Security, Applicability: A Surveycited by 14
  7. 2024GI-NAS: Boosting Gradient Inversion Attacks Through Adaptive Neural Architecture Searchcited by 14
  8. 2024FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneitycited by 13
  9. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  10. 2024Task Arithmetic Through The Lens Of One-Shot Federated Learningcited by 12
  11. 2025Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learningcited by 12
  12. 2023pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharingcited by 9
38 more
  1. 2024Enhancing Trust and Privacy in Distributed Networks: A Comprehensive Survey on Blockchain-based Federated Learningcited by 9
  2. 2024Decentralized Directed Collaboration for Personalized Federated Learningcited by 9
  3. 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applicationscited by 8
  4. 2024RBLA: Rank-Based-LoRA-Aggregation for Fine-tuning Heterogeneous Models in FLaaScited by 8
  5. 2025Federated Learning for Diffusion Modelscited by 7
  6. 2025HtFLlib: A Comprehensive Heterogeneous Federated Learning Library and Benchmarkcited by 7
  7. 2023Coordination-free Decentralised Federated Learning on Complex Networks: Overcoming Heterogeneitycited by 6
  8. 2024Communication-Efficient Federated Low-Rank Update Algorithm and its Connection to Implicit Regularizationcited by 6
  9. 2025Patient-Level Anatomy Meets Scanning-Level Physics: Personalized Federated Low-Dose CT Denoising Empowered by Large Language Modelcited by 6
  10. 2025FNBench: Benchmarking Robust Federated Learning against Noisy Labelscited by 6
  11. 2024From Optimization to Generalization: Fair Federated Learning against Quality Shift via Inter-Client Sharpness Matchingcited by 5
  12. 2024Worldwide Federated Training of Language Modelscited by 5
  13. 2025Robust Asymmetric Heterogeneous Federated Learning with Corrupted Clientscited by 5
  14. 2025From Laboratory to Real World: A New Benchmark Towards Privacy-Preserved Visible-Infrared Person Re-Identificationcited by 5
  15. 2024DFML: Decentralized Federated Mutual Learningcited by 4
  16. 2024Logit Calibration and Feature Contrast for Robust Federated Learning on Non-IID Datacited by 4
  17. 2024Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspectivecited by 4
  18. 2024Federated Fairness Analytics: Quantifying Fairness in Federated Learningcited by 4
  19. 2025Homophily Heterogeneity Matters in Graph Federated Learning: A Spectrum Sharing and Complementing Perspectivecited by 4
  20. 2025FedCanon: Non-Convex Composite Federated Learning with Efficient Proximal Operation on Heterogeneous Datacited by 4
  21. 2024Bridging Data Islands: Geographic Heterogeneity-Aware Federated Learning for Collaborative Remote Sensing Semantic Segmentationcited by 3
  22. 2024FedCoSR: Personalized Federated Learning with Contrastive Shareable Representations for Label Heterogeneity in Non-IID Datacited by 3
  23. 2024Faster Convergence on Heterogeneous Federated Edge Learning: An Adaptive Clustered Data Sharing Approachcited by 3
  24. 2024Federated Automatic Latent Variable Selection in Multi-output Gaussian Processescited by 3
  25. 2024Tackling Data Heterogeneity in Federated Time Series Forecastingcited by 3
  26. 2025Federated EndoViT: Pretraining Vision Transformers via Federated Learning on Endoscopic Image Collectionscited by 3
  27. 2025An Empirical Study of Federated Prompt Learning for Vision Language Modelcited by 3
  28. 2023Privacy and Accuracy Implications of Model Complexity and Integration in Heterogeneous Federated Learningcited by 2
  29. 2024Federated Learning with Heterogeneous Data Handling for Robust Vehicular Object Detectioncited by 2
  30. 2024On the Volatility of Shapley-Based Contribution Metrics in Federated Learningcited by 2
  31. 2024Geometrically Inspired Kernel Machines for Collaborative Learning Beyond Gradient Descentcited by 2
  32. 2024Addressing Heterogeneity in Federated Learning: Challenges and Solutions for a Shared Production Environmentcited by 2
  33. 2025Bad-PFL: Exploring Backdoor Attacks against Personalized Federated Learningcited by 2
  34. 2025FedPref: Federated Learning Across Heterogeneous Multi-objective Preferencescited by 2
  35. 2025A survey on Clustered Federated Learning: Taxonomy, Analysis and Applicationscited by 2
  36. 2025Fine-Tuning Foundation Models with Federated Learning for Privacy Preserving Medical Time Series Forecastingcited by 2
  37. 2025Preconditioned Inexact Stochastic ADMM for Deep Modelcited by 2
  38. 2025Adaptive UAV-Assisted Hierarchical Federated Learning: Optimizing Energy, Latency, and Resilience for Dynamic Smart IoTcited by 2

50 most cited of 161 All 161 (JSON)