mlsanityData to 14 Oct 2026

Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directions

arXiv 2411.12377 ↗First posted 19 Nov 2024cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Recent advances in machine learning have highlighted Federated Learning (FL) as a promising approach that enables multiple distributed users (so-called clients) to collectively train ML models without sharing their private data. While this privacy-preserving method shows potential, it struggles when data across clients is not independent and identically distributed (non-IID) data. The latter remains an unsolved challenge that can result in poorer model performance and slower training times. Despite the significance of non-IID data in FL, there is a lack of consensus among researchers about its classification and quantification. This technical survey aims to fill that gap by providing a detailed taxonomy for non-IID data, partition protocols, and metrics to quantify data heterogeneity. Additionally, we describe popular solutions to address non-IID data and standardized frameworks employed in FL with heterogeneous data. Based on our state-of-the-art survey, we present key lessons learned and suggest promising future research directions.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 121 of its 266 references that are in the map. First posted on arXiv 19 Nov 2024.

Builds on

121 of 266 references.

References · 121

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2018Federated Learning with Non-IID Datacited by 1,108
  5. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  6. 2018LEAF: A Benchmark for Federated Settingscited by 792
  7. 2021Model-Contrastive Federated Learningcited by 766
  8. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  9. 2021Towards Personalized Federated Learningcited by 464
  10. 2019Agnostic Federated Learningcited by 456
  11. 2021Federated Learning on Non-IID Data: A Surveycited by 346
  12. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 305
38 more
  1. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 300
  2. 2021Federated Learning for Internet of Things: A Comprehensive Surveycited by 252
  3. 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
  4. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 198
  5. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  6. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  7. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  8. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 144
  9. 2020Federated Visual Classification with Real-World Data Distributioncited by 128
  10. 2021FairFed: Enabling Group Fairness in Federated Learningcited by 118
  11. 2019BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learningcited by 117
  12. 2021Decentralized Federated Averagingcited by 114
  13. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  14. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 109
  15. 2022Vertical Federated Learning: Concepts, Advances and Challengescited by 108
  16. 2020Robust Federated Learning: The Case of Affine Distribution Shiftscited by 102
  17. 2021Asynchronous Federated Learning on Heterogeneous Devices: A Surveycited by 93
  18. 2022Federated Learning on Non-IID Graphs via Structural Knowledge Sharingcited by 88
  19. 2023Multimodal Federated Learning via Contrastive Representation Ensemblecited by 86
  20. 2021Federated Learning: Opportunities and Challengescited by 84
  21. 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 81
  22. 2021Differentially Private Federated Learning on Heterogeneous Datacited by 74
  23. 2020Over-the-Air Federated Learning from Heterogeneous Datacited by 73
  24. 2022Tackling Data Heterogeneity in Federated Learning with Class Prototypescited by 67
  25. 2021HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Imagescited by 64
  26. 2020LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasetscited by 60
  27. 2021OpenFL: An open-source framework for Federated Learningcited by 58
  28. 2019On Safeguarding Privacy and Security in the Framework of Federated Learningcited by 55
  29. 2021FedLab: A Flexible Federated Learning Frameworkcited by 55
  30. 2023FedFed: Feature Distillation against Data Heterogeneity in Federated Learningcited by 54
  31. 2023Federated Continual Learning via Knowledge Fusion: A Surveycited by 53
  32. 2023FedMultimodal: A Benchmark For Multimodal Federated Learningcited by 51
  33. 2022ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsitycited by 45
  34. 2020From Federated Learning to Federated Neural Architecture Search: A Surveycited by 42
  35. 2023Prompt Federated Learning for Weather Forecasting: Toward Foundation Models on Meteorological Datacited by 40
  36. 2021Node Selection Toward Faster Convergence for Federated Learning on Non-IID Datacited by 39
  37. 2022Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imagingcited by 39
  38. 2022Cross-Silo Federated Learning: Challenges and Opportunitiescited by 38

50 most cited of 121 All 121 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 7.

13 papers.

Sort:

By area:

  1. 2025FBFL: A Field-Based Coordination Approach for Data Heterogeneity in Federated Learningcited by 5
  2. 2024Asynchronous Federated Learning: A Scalable Approach for Decentralized Machine Learningcited by 3
  3. 2025Federated Learning With L0 Constraint Via Probabilistic Gates For Sparsitycited by 1
  4. 2026When More Parameters Hurt: Foundation Model Priors Amplify Worst-Client Disparity Under Extreme Federated Heterogeneitycited by 1
  5. 2025FeDa4Fair: Client-Level Federated Datasets for Fairness Evaluationcited by 0
  6. 2025Training Together, Diagnosing Better: Federated Learning for Collagen VI-Related Dystrophiescited by 0
  7. 2025Clust-PSI-PFL: A Population Stability Index Approach for Clustered Non-IID Personalized Federated Learningcited by 0
  8. 2026Task2vec Readiness: Diagnostics for Federated Learning from Pre-Training Embeddingscited by 0
  9. 2026AlignFed: Alignment-Aware Asynchronous Federated Fine-Tuning for Large Language Models in Heterogeneous Edge Environmentscited by 0
  10. 2026Entropy-Regularized Probabilistic Gates for Sparse Model Discovery in Scarce-Data Federated Learningcited by 0
  11. 2026Benchmarking Federated Learning and Knowledge Distillation for Point Cloud Classificationcited by 0
  12. 2026On the Effectiveness of Adaptation Strategies for VLM-Based Federated Learning in Remote Sensingcited by 0
1 more
  1. 2026Perplexity Predicts Protection: Choosing Pretrained Backbones for Worst-Client Fairness in Federated Parameter-Efficient Fine-Tuningcited by 0