mlsanityData to 7 Oct 2026

From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learning

arXiv 2606.10595 ↗First posted 9 Jun 2026cs.CR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) has emerged as a promising solution for data hunger in centralized learning. This paradigm enables privacy with multiple clients to train a shared-task model collaboratively without exposing their local data. While being a key component in any learning system, data is also a primary source of vulnerabilities and challenges, and a major determinant of a stable and well-converged training. Existing FL reviews describe general foundations, security practices, opportunities, challenges, and applications, without delving into diverse aspects of data and considering problems from the data perspective. They rarely provide a data-lens synthesis that links concrete data properties, split protocols, and defenses to convergence speed and stability. This survey fills that gap with three advances. First, we analyze non-IID into measurable traits and rank their influence on convergence as strong, medium, or light, explaining the mechanisms behind each and reconciling evidence across images, texts, and graphs. Second, we connect experimental splitting practices to the real phenomena they emulate, expose the artifacts they introduce, and show how those artifacts affect target accuracy. Third, we analyze how data-related vulnerabilities and their proposed defenses affect convergence, reporting performance under clean and adversarial conditions to make the convergence-robustness trade-off explicit. To our knowledge, this is the first survey to provide a complete understanding of data-related challenges that govern FL. With clear takeaways distilled for each concern, our work serves as actionable guidance, helping practitioners design their system with predictable convergence and stability.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 106 of its 200 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

106 of 200 references.

References · 106

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  6. 2018Experience Replay for Continual Learningcited by 1,074
  7. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  8. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  9. 2021Membership Inference Attacks From First Principlescited by 839
  10. 2019Towards Federated Learning at Scale: System Designcited by 826
  11. 2018How To Backdoor Federated Learningcited by 745
  12. 2021Federated Learning Based on Dynamic Regularizationcited by 555
38 more
  1. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  2. 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
  3. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  4. 2020Attack of the Tails: Yes, You Really Can Backdoor Federated Learningcited by 307
  5. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
  6. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
  7. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 266
  8. 2021Federated Learning for Internet of Things: A Comprehensive Surveycited by 247
  9. 2019MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examplescited by 183
  10. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  11. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
  12. 2019Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inferencecited by 168
  13. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 135
  14. 2022FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settingscited by 121
  15. 2022FLDetector: Defending Federated Learning Against Model Poisoning Attacks via Detecting Malicious Clientscited by 119
  16. 2021Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learningcited by 116
  17. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 112
  18. 2022Vertical Federated Learning: Concepts, Advances and Challengescited by 104
  19. 2022Improving Generalization in Federated Learning by Seeking Flat Minimacited by 98
  20. 2021Asynchronous Federated Learning on Heterogeneous Devices: A Surveycited by 93
  21. 2020Convergence Time Optimization for Federated Learning over Wireless Networkscited by 81
  22. 2024Continual Learning and Catastrophic Forgettingcited by 73
  23. 2022Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillationcited by 72
  24. 2022Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learningcited by 71
  25. 2023FedDisco: Federated Learning with Discrepancy-Aware Collaborationcited by 66
  26. 2021From Distributed Machine Learning to Federated Learning: A Surveycited by 65
  27. 2020Data Poisoning Attacks on Federated Machine Learningcited by 58
  28. 2020Federated Learning with Only Positive Labelscited by 55
  29. 2023FedFed: Feature Distillation against Data Heterogeneity in Federated Learningcited by 54
  30. 2023FedMultimodal: A Benchmark For Multimodal Federated Learningcited by 51
  31. 2021Flexible Clustered Federated Learning for Client-Level Data Distribution Shiftcited by 47
  32. 2021SparseFed: Mitigating Model Poisoning Attacks in Federated Learning with Sparsificationcited by 46
  33. 2023A Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Taskscited by 41
  34. 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
  35. 2023Personalized Federated Learning under Mixture of Distributionscited by 37
  36. 2024Towards Efficient Replay in Federated Incremental Learningcited by 33
  37. 2022Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reductioncited by 32
  38. 2023On the Convergence of Federated Averaging with Cyclic Client Participationcited by 31

50 most cited of 106 All 106 (JSON)

Built on by

Not cited yet.