mlsanityData to 7 Oct 2026

Advances in Robust Federated Learning: A Survey with Heterogeneity Considerations

arXiv 2405.09839 ↗First posted 16 May 2024cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

In the field of heterogeneous federated learning (FL), the key challenge is to efficiently and collaboratively train models across multiple clients with different data distributions, model structures, task objectives, computational capabilities, and communication resources. This diversity leads to significant heterogeneity, which increases the complexity of model training. In this paper, we first outline the basic concepts of heterogeneous federated learning and summarize the research challenges in federated learning in terms of five aspects: data, model, task, device, and communication. In addition, we explore how existing state-of-the-art approaches cope with the heterogeneity of federated learning, and categorize and review these approaches at three different levels: data-level, model-level, and architecture-level. Subsequently, the paper extensively discusses privacy-preserving strategies in heterogeneous federated learning environments. Finally, the paper discusses current open issues and directions for future research, aiming to promote the further development of heterogeneous federated learning.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 116 of its 208 references that are in the map. First posted on arXiv 16 May 2024.

Builds on

116 of 208 references.

References · 116

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2016Deep Learning with Differential Privacycited by 2,682
  5. 2019Deep Leakage from Gradientscited by 1,004
  6. 2021Model-Contrastive Federated Learningcited by 762
  7. 2018How To Backdoor Federated Learningcited by 745
  8. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  9. 2017Federated Multi-Task Learningcited by 669
  10. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  11. 2017Renyi Differential Privacycited by 643
  12. 2020Personalized Federated Learning with Moreau Envelopescited by 612
38 more
  1. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  2. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  3. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  4. 2019Federated Learning with Personalization Layerscited by 533
  5. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  6. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  7. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  8. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  9. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  10. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  11. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
  12. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  13. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
  14. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  15. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  16. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  17. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  18. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 285
  19. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  20. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 266
  21. 2021Federated Learning for Internet of Things: A Comprehensive Surveycited by 247
  22. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  23. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
  24. 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
  25. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  26. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  27. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 185
  28. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  29. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  30. 2020Privacy and Robustness in Federated Learning: Attacks and Defensescited by 163
  31. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  32. 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 154
  33. 2019Overcoming Forgetting in Federated Learning on Non-IID Datacited by 120
  34. 2021FairFed: Enabling Group Fairness in Federated Learningcited by 114
  35. 2022Federated Learning with Partial Model Personalizationcited by 114
  36. 2020Feature Inference Attack on Model Predictions in Vertical Federated Learningcited by 105
  37. 2020Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspectivecited by 102
  38. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102

50 most cited of 116 All 116 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 6.

12 papers.

  1. 2024Threats and Defenses in Federated Learning Life Cycle: A Comprehensive Survey and Challengescited by 14
  2. 2025Geometric Prior-Guided Federated Prompt Calibrationcited by 2
  3. 2025FedTeddi: Temporal Drift and Divergence Aware Scheduling for Timely Federated Edge Learningcited by 1
  4. 2025pFedBBN: A Personalized Federated Test-Time Adaptation with Balanced Batch Normalization for Class-Imbalanced Datacited by 1
  5. 2026Distribution Alignment for One-Shot Federated Learning via Optimal Transportcited by 1
  6. 2025From Privacy to Trust in the Agentic Era: A Taxonomy of Challenges in Trustworthy Federated Learning Through the Lens of Trust Report 2.0cited by 0
  7. 2025Energy-Efficient Federated Learning via Adaptive Encoder Freezing for MRI-to-CT Conversion: A Green AI-Guided Researchcited by 0
  8. 2026Beyond Fixed Rounds: Data-Free Early Stopping for Practical Federated Learningcited by 0
  9. 2026MLFCIL: A Multi-Level Forgetting Mitigation Framework for Federated Class-Incremental Learning in LEO Satellitescited by 0
  10. 2026Representation-Aligned Multi-Scale Personalization for Federated Learningcited by 0
  11. 2026FedCARE: A Multi-Objective Personalised Federated Learning Framework for Smart Healthcarecited by 0
  12. 2026Prototype-guided Bilateral Alignment Multimodal Federated Learningcited by 0