mlsanityData to 14 Oct 2026

A Survey on Heterogeneous Federated Learning

arXiv 2210.04505 ↗First posted 10 Oct 2022cs.LGcited by 42

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) has been proposed to protect data privacy and virtually assemble the isolated data silos by cooperatively training models among organizations without breaching privacy and security. However, FL faces heterogeneity from various aspects, including data space, statistical, and system heterogeneity. For example, collaborative organizations without conflict of interest often come from different areas and have heterogeneous data from different feature spaces. Participants may also want to train heterogeneous personalized local models due to non-IID and imbalanced data distribution and various resource-constrained devices. Therefore, heterogeneous FL is proposed to address the problem of heterogeneity in FL. In this survey, we comprehensively investigate the domain of heterogeneous FL in terms of data space, statistical, system, and model heterogeneity. We first give an overview of FL, including its definition and categorization. Then, We propose a precise taxonomy of heterogeneous FL settings for each type of heterogeneity according to the problem setting and learning objective. We also investigate the transfer learning methodologies to tackle the heterogeneity in FL. We further present the applications of heterogeneous FL. Finally, we highlight the challenges and opportunities and envision promising future research directions toward new framework design and trustworthy approaches.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Builds on 118 of its 223 references that are in the map. First posted on arXiv 10 Oct 2022.

Builds on

118 of 223 references.

References · 118

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2020Knowledge Distillation: A Surveycited by 1,603
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  6. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  7. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  8. 2018Federated Learning with Non-IID Datacited by 1,108
  9. 2019Relational Knowledge Distillationcited by 889
  10. 2019Contrastive Representation Distillationcited by 748
  11. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  12. 2017Federated Multi-Task Learningcited by 672
38 more
  1. 2017Deep Mutual Learningcited by 628
  2. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 564
  3. 2019Federated Learning with Personalization Layerscited by 541
  4. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  5. 2013Do Deep Nets Really Need to be Deep?cited by 474
  6. 2021Towards Personalized Federated Learningcited by 464
  7. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 459
  8. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  9. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
  10. 2019Federated Machine Learning: Concept and Applicationscited by 412
  11. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 395
  12. 2015Return of Frustratingly Easy Domain Adaptationcited by 391
  13. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  14. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
  15. 2020Adaptive Personalized Federated Learningcited by 358
  16. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
  17. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 352
  18. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  19. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  20. 2019Robust Aggregation for Federated Learningcited by 317
  21. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 283
  22. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  23. 2019Asynchronous Federated Optimizationcited by 267
  24. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 243
  25. 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 210
  26. 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
  27. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 204
  28. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
  29. 2018Knowledge Distillation by On-the-Fly Native Ensemblecited by 186
  30. 2017Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryptioncited by 176
  31. 2019SecureBoost: A Lossless Federated Learning Frameworkcited by 176
  32. 2015Cross Modal Distillation for Supervision Transfercited by 175
  33. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 163
  34. 2017A Survey on Homomorphic Encryption Schemes: Theory and Implementationcited by 156
  35. 2019Federated Evaluation of On-device Personalizationcited by 147
  36. 2019Federated Adversarial Domain Adaptationcited by 140
  37. 2019Zero-shot Knowledge Transfer via Adversarial Belief Matchingcited by 129
  38. 2019Online Knowledge Distillation with Diverse Peerscited by 128

50 most cited of 118 All 118 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 13; 2024: 13; 2025: 10; 2026: 6.

42 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
  2. 2023DiLoCo: Distributed Low-Communication Training of Language Modelscited by 121
  3. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 84
  4. 2023Cross-Silo Prototypical Calibration for Federated Learning with Non-IID Datacited by 35
  5. 2024Federated Large Language Models: Current Progress and Future Directionscited by 27
  6. 2024Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchorscited by 21
  7. 2023Multimodal Federated Learning in Healthcare: a Reviewcited by 13
  8. 2024FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneitycited by 13
  9. 2024Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcarecited by 12
  10. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  11. 2023FedYolo: Augmenting Federated Learning with Pretrained Transformerscited by 9
  12. 2024DapperFL: Domain Adaptive Federated Learning with Model Fusion Pruning for Edge Devicescited by 9
30 more
  1. 2024FedLPS: Heterogeneous Federated Learning for Multiple Tasks with Local Parameter Sharingcited by 8
  2. 2024Federated Graph Condensation with Information Bottleneck Principlescited by 8
  3. 2023An effective theory of collective deep learningcited by 7
  4. 2023Aggregating Intrinsic Information to Enhance BCI Performance through Federated Learningcited by 6
  5. 2023Internal Cross-layer Gradients for Extending Homogeneity to Heterogeneity in Federated Learningcited by 5
  6. 2025Federated Out-of-Distribution Generalization: A Causal Augmentation Viewcited by 4
  7. 2023UPFL: Unsupervised Personalized Federated Learning towards New Clientscited by 3
  8. 2025Entangled Mean Estimation in High-Dimensionscited by 3
  9. 2025Global Intervention and Distillation for Federated Out-of-Distribution Generalizationcited by 3
  10. 2025Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environmentscited by 2
  11. 2023Personalized Federated Learning via Amortized Bayesian Meta-Learningcited by 1
  12. 2024Dataset Dictionary Learning in a Wasserstein Space for Federated Domain Adaptationcited by 1
  13. 2025Conditioning on Local Statistics for Scalable Heterogeneous Federated Learningcited by 1
  14. 2025Data-Free Black-Box Federated Learning via Zeroth-Order Gradient Estimationcited by 1
  15. 2025DP2FL: Dual Prompt Personalized Federated Learning in Foundation Modelscited by 1
  16. 2026Decision-Focused Federated Learning Under Heterogeneous Objectives and Constraintscited by 1
  17. 2023Heterogeneous Federated Learning via Personalized Generative Networkscited by 0
  18. 2023Evaluating Multi-Global Server Architecture for Federated Learningcited by 0
  19. 2024MAPL: Model Agnostic Peer-to-peer Learningcited by 0
  20. 2024UniFed: A Universal Federation of a Mixture of Highly Heterogeneous Medical Image Classification Taskscited by 0
  21. 2024Tailored Federated Learning: Leveraging Direction Regulation & Knowledge Distillationcited by 0
  22. 2024Federated Learning for Discrete Optimal Transport with Large Population under Incomplete Informationcited by 0
  23. 2025FedStrategist: A Meta-Learning Framework for Adaptive and Robust Aggregation in Federated Learningcited by 0
  24. 2025VGS-ATD: Robust Distributed Learning for Multi-Label Medical Image Classification Under Heterogeneous and Imbalanced Conditionscited by 0
  25. 2025FedAPA: Federated Learning with Adaptive Prototype Aggregation Toward Heterogeneous Wi-Fi CSI-based Crowd Countingcited by 0
  26. 2026Self-Supervised Federated Learning under Data Heterogeneity for Label-Scarce Diatom Classificationcited by 0
  27. 2026Automating aggregation strategy selection in federated learningcited by 0
  28. 2026Decentralized Learning via Random Walk with Jumpscited by 0
  29. 2026Federated Rule Ensemble Method in Medical Datacited by 0
  30. 2026LLM-Based Schema-Aware Split Learning for Privacy-Preserving Mental Distress Prediction Across Heterogeneous Surveyscited by 0