Data to 14 Oct 2026

Federated Learning with Non-IID Data

arXiv 1806.00582 ↗First posted 2 Jun 2018cs.LG stat.MLcited by 1,108

Not on the map. Why?

Cited by 22 in Knowledge distillation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Federated learning enables resource-constrained edge compute devices, such as mobile phones and IoT devices, to learn a shared model for prediction, while keeping the training data local. This decentralized approach to train models provides privacy, security, regulatory and economic benefits. In this work, we focus on the statistical challenge of federated learning when local data is non-IID. We first show that the accuracy of federated learning reduces significantly, by up to 55% for neural networks trained for highly skewed non-IID data, where each client device trains only on a single class of data. We further show that this accuracy reduction can be explained by the weight divergence, which can be quantified by the earth mover's distance (EMD) between the distribution over classes on each device and the population distribution. As a solution, we propose a strategy to improve training on non-IID data by creating a small subset of data which is globally shared between all the edge devices. Experiments show that accuracy can be increased by 30% for the CIFAR-10 dataset with only 5% globally shared data.

Key facts. Cited by 1,108 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 2; 2021: 200; 2022: 203; 2023: 231; 2024: 197; 2025: 162; 2026: 113.

1,108 papers.

Sort:

By area:

2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
2021Federated Learning Based on Dynamic Regularizationcited by 562
2021Towards Personalized Federated Learningcited by 464
2021Federated Learning on Non-IID Data: A Surveycited by 346
2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
2021Personalized Federated Learning using Hypernetworkscited by 212
2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
2021Federated Learning for Smart Healthcare: A Surveycited by 184
2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 182
2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
2024FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptationscited by 146
2021Federated Learning for Open Bankingcited by 141

38 more

2022PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Modelcited by 134
2022FedALA: Adaptive Local Aggregation for Personalized Federated Learningcited by 122
2021Federated Unlearning via Class-Discriminative Pruningcited by 115
2021Federated Graph Classification over Non-IID Graphscited by 114
2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 113
2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 109
2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
2023Stochastic Controlled Averaging for Federated Learning with Communication Compressioncited by 101
2021DENSE: Data-Free One-Shot Federated Learningcited by 100
2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 99
2022Improving Generalization in Federated Learning by Seeking Flat Minimacited by 98
2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
2021CRFL: Certifiably Robust Federated Learning against Backdoor Attackscited by 91
2022Trustworthy Graph Neural Networks: Aspects, Methods and Trendscited by 89
2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
2023Multimodal Federated Learning via Contrastive Representation Ensemblecited by 86
2023Decentralized Federated Learning: A Survey and Perspectivecited by 86
2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 84
2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 81
2024Selective Aggregation for Low-Rank Adaptation in Federated Learningcited by 80
2021FedCM: Federated Learning with Client-level Momentumcited by 77
2021Personalized Federated Learning with Gaussian Processescited by 76
2022When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methodscited by 75
2022Divergence-aware Federated Self-Supervised Learningcited by 73
2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
2022Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learningcited by 71
2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 69
2021Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Datacited by 69
2021Collaborative Unsupervised Visual Representation Learning from Decentralized Datacited by 69
2023FedDisco: Federated Learning with Discrepancy-Aware Collaborationcited by 67
2022FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learningcited by 66
2023TARGET: Federated Class-Continual Learning via Exemplar-Free Distillationcited by 65
2021Towards Fair Federated Learning with Zero-Shot Data Augmentationcited by 64
2021HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Imagescited by 64
2021What Do We Mean by Generalization in Federated Learning?cited by 62
2023Better Generative Replay for Continual Federated Learningcited by 55
2023FedFed: Feature Distillation against Data Heterogeneity in Federated Learningcited by 54

50 most cited of 1,108 All 1,108 (JSON)