mlsanityData to 7 Oct 2026

A Thorough Assessment of the Non-IID Data Impact in Federated Learning

arXiv 2503.17070 ↗First posted 21 Mar 2025cs.LG cs.AI stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) allows collaborative machine learning (ML) model training among decentralized clients' information, ensuring data privacy. The decentralized nature of FL deals with non-independent and identically distributed (non-IID) data. This open problem has notable consequences, such as decreased model performance and more significant convergence times. Despite its importance, experimental studies systematically addressing all types of data heterogeneity (a.k.a. non-IIDness) remain scarce. We aim to fill this gap by assessing and quantifying the non-IID effect through a thorough empirical analysis. We use the Hellinger Distance (HD) to measure differences in distribution among clients. Our study benchmarks four state-of-the-art strategies for handling non-IID data, including label, feature, quantity, and spatiotemporal skewness, under realistic and controlled conditions. This is the first comprehensive analysis of the spatiotemporal skew effect in FL. Our findings highlight the significant impact of label and spatiotemporal skew non-IID types on FL model performance, with notable performance drops occurring at specific HD thresholds. Additionally, the FL performance is heavily affected mainly when the non-IIDness is extreme. Thus, we provide recommendations for FL research to tackle data heterogeneity effectively. Our work represents the most extensive examination of non-IIDness in FL, offering a robust foundation for future research.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 38 of its 84 references that are in the map. First posted on arXiv 21 Mar 2025.

Builds on

38 of 84 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  6. 2019Advances and Open Problems in Federated Learningcited by 3,021
  7. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  8. 2021Model-Contrastive Federated Learningcited by 762
  9. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  10. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  11. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  12. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
26 more
  1. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  2. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
  3. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 196
  4. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 185
  5. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 135
  6. 2022FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settingscited by 121
  7. 2021Local Learning Matters: Rethinking Data Heterogeneity in Federated Learningcited by 114
  8. 2020Fast-Convergent Federated Learning with Adaptive Weightingcited by 67
  9. 2019On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract meanscited by 61
  10. 2021FedLab: A Flexible Federated Learning Frameworkcited by 54
  11. 2023Federated Continual Learning via Knowledge Fusion: A Surveycited by 53
  12. 2022Do Gradient Inversion Attacks Make Federated Learning Unsafe?cited by 44
  13. 2021Non-IID data and Continual Learning processes in Federated Learning: A long road aheadcited by 36
  14. 2023Why Batch Normalization Damage Federated Learning on Non-IID Data?cited by 29
  15. 2021ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Trainingcited by 27
  16. 2022Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarkscited by 24
  17. 2021Federated Learning with Taskonomy for Non-IID Datacited by 21
  18. 2021Privacy and Trust Redefined in Federated Machine Learningcited by 13
  19. 2022On Approximating Total Variation Distancecited by 13
  20. 2022Application of federated learning techniques for arrhythmia classification using 12-lead ECG signalscited by 9
  21. 2023An Empirical Study of Federated Learning on IoT-Edge Devices: Resource Allocation and Heterogeneitycited by 5
  22. 2023Jointly Exploring Client Drift and Catastrophic Forgetting in Dynamic Learningcited by 5
  23. 2024A Universal Metric of Dataset Similarity for Cross-silo Federated Learningcited by 4
  24. 2023Digital Ethics in Federated Learningcited by 2
  25. 2024The Effects of Data Imbalance Under a Federated Learning Approach for Credit Risk Forecastingcited by 2
  26. 2025STHFL: Spatio-Temporal Heterogeneous Federated Learningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026CRAD: Class-wise Reliability-Aware Distillation for Decentralized Heterogeneous Federated Learningcited by 1
  2. 2026Beyond Perfect Scores: Proof-by-Contradiction for Trustworthy Machine Learningcited by 0
  3. 2026FedLECC: Cluster- and Loss-Guided Client Selection for Federated Learning under Non-IID Datacited by 0
  4. 2026SatDL: Jointly Optimizing Data Redistribution and Training for Satellite-Based Distributed Learningcited by 0
  5. 2026CALM: Class-wise Agreement and Label-gated Disagreement Modulation for Decentralized Federated Learningcited by 0