Federated Learning on Non-IID Data: A Survey
Not on the map. Why?
Abstract (arXiv)
Federated learning is an emerging distributed machine learning framework for privacy preservation. However, models trained in federated learning usually have worse performance than those trained in the standard centralized learning mode, especially when the training data are not independent and identically distributed (Non-IID) on the local devices. In this survey, we pro-vide a detailed analysis of the influence of Non-IID data on both parametric and non-parametric machine learning models in both horizontal and vertical federated learning. In addition, cur-rent research work on handling challenges of Non-IID data in federated learning are reviewed, and both advantages and disadvantages of these approaches are discussed. Finally, we suggest several future research directions before concluding the paper.
Key facts. Cited by 345 papers in the map (data through 7 Oct 2026). Builds on 110 of its 190 references that are in the map. First posted on arXiv 12 Jun 2021.
Builds on
110 of 190 references.
References · 110
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2020Knowledge Distillation: A Surveycited by 1,578
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
- 2018Group Normalizationcited by 1,514
38 more
- 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
- 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
- 2018Federated Learning with Non-IID Datacited by 1,099
- 2020Adaptive Federated Optimizationcited by 1,036
- 2019Deep Leakage from Gradientscited by 1,004
- 2018On First-Order Meta-Learning Algorithmscited by 1,003
- 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
- 2019Towards Federated Learning at Scale: System Designcited by 826
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
- 2017Federated Multi-Task Learningcited by 669
- 2017Deep Mutual Learningcited by 621
- 2020Personalized Federated Learning with Moreau Envelopescited by 612
- 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
- 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 564
- 2020Federated Learning with Matched Averagingcited by 564
- 2019Federated Learning with Personalization Layerscited by 533
- 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
- 2020An Efficient Framework for Clustered Federated Learningcited by 505
- 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
- 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
- 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
- 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
- 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
- 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
- 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
- 2020iDLG: Improved Deep Leakage from Gradientscited by 367
- 2020Adaptive Personalized Federated Learningcited by 353
- 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 326
- 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
- 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
- 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
- 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
- 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
- 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 253
- 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 242
- 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
- 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
- 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
50 most cited of 110 All 110 (JSON)
Built on by
Citing papers per year: 2021: 5; 2022: 46; 2023: 68; 2024: 94; 2025: 92; 2026: 40.
345 papers.
Sort:
By area:
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
- 2022NOTE: Robust Continual Test-time Adaptation Against Temporal Correlationcited by 150
- 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
- 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
- 2024Selective Aggregation for Low-Rank Adaptation in Federated Learningcited by 80
- 2023Universal Test-time Adaptation through Weight Ensembling, Diversity Weighting, and Prior Correctioncited by 65
- 2022Client Selection in Federated Learning: Principles, Challenges, and Opportunitiescited by 62
- 2023Better Generative Replay for Continual Federated Learningcited by 53
- 2023Federated Conformal Predictors for Distributed Uncertainty Quantificationcited by 46
- 2022Cross-Silo Federated Learning: Challenges and Opportunitiescited by 38
- 2022Federated Learning for Medical Applications: A Taxonomy, Current Trends, Challenges, and Future Research Directionscited by 34
- 2024An Aggregation-Free Federated Learning for Tackling Data Heterogeneitycited by 33
38 more
- 2022Towards Federated Clustering: A Federated Fuzzy c-Means Algorithm (FFCM)cited by 29
- 2022FedKL: Tackling Data Heterogeneity in Federated Reinforcement Learning by Penalizing KL Divergencecited by 29
- 2022A Critical Review on the Use (and Misuse) of Differential Privacy in Machine Learningcited by 27
- 2021Personalized Federated Learning with Adaptive Batchnorm for Healthcarecited by 26
- 2022CrowdGuard: Federated Backdoor Detection in Federated Learningcited by 26
- 2023Privacy and Fairness in Federated Learning: on the Perspective of Trade-offcited by 26
- 2024FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decompositioncited by 26
- 2022Federated Learning with GAN-based Data Synthesis for Non-IID Clientscited by 25
- 2023Selective Knowledge Sharing for Privacy-Preserving Federated Distillation without A Good Teachercited by 25
- 2022Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matchingcited by 24
- 2022FRAug: Tackling Federated Learning with Non-IID Features via Representation Augmentationcited by 24
- 2023A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacycited by 24
- 2023FS-Real: Towards Real-World Cross-Device Federated Learningcited by 22
- 2023Federated Learning for Computer Visioncited by 22
- 2023Phoenix: A Federated Generative Diffusion Modelcited by 21
- 2023A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiencycited by 21
- 2022CalFAT: Calibrated Federated Adversarial Training with Label Skewnesscited by 20
- 2023MP-FedCL: Multiprototype Federated Contrastive Learning for Edge Intelligencecited by 20
- 2022Vertical Federated Learning: A Structured Literature Reviewcited by 17
- 2023Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Surveycited by 17
- 2023A Survey for Federated Learning Evaluations: Goals and Measurescited by 16
- 2024OpenFGL: A Comprehensive Benchmark for Federated Graph Learningcited by 16
- 2022Federated Learning for 5G Base Station Traffic Forecastingcited by 15
- 2023Accelerating Hybrid Federated Learning Convergence under Partial Participationcited by 15
- 2025Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Modelcited by 15
- 2022Federated Graph Machine Learning: A Survey of Concepts, Techniques, and Applicationscited by 14
- 2024Federated Distillation: A Surveycited by 14
- 2022A Survey on Computationally Efficient Neural Architecture Searchcited by 12
- 2023A Survey on Class Imbalance in Federated Learningcited by 12
- 2023Federated Learning without Full Labels: A Surveycited by 12
- 2023FedCiR: Client-Invariant Representation Learning for Federated Non-IID Featurescited by 12
- 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
- 2024Green Federated Learning: A new era of Green Aware AIcited by 12
- 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 12
- 2024One-shot Federated Learning via Synthetic Distiller-Distillate Communicationcited by 12
- 2022SHED: A Newton-type algorithm for federated learning based on incremental Hessian eigenvector sharingcited by 11
- 2023Dealing With Heterogeneous 3D MR Knee Images: A Federated Few-Shot Learning Method With Dual Knowledge Distillationcited by 11
- 2023QuickDrop: Efficient Federated Unlearning by Integrated Dataset Distillationcited by 11
50 most cited of 345 All 345 (JSON)
- 2026FedMAD: Modulation-Aware Directional Aggregation for Federated Learning in Remote Sensing Image Classification30 Sep 2026
- 2026Making Cross-Continental Federated Learning Repeatable with FLIP: a Multi-Application Study28 Sep 2026
- 2026Federated Deep Clustering Networks for High-Dimensional and Heterogeneous Data18 Sep 2026
- 2026Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation15 Sep 2026
- 2026Decentralized Gossip Learning and Federated Averaging for Histopathology Image Classification15 Sep 2026
- 2026SatDL: Jointly Optimizing Data Redistribution and Training for Satellite-Based Distributed Learning25 Aug 2026
- 2026FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting15 Aug 2026
- 2026Beyond Parameter Space: NTK-Guided Personalized Aggregation for Robust Federated Learning12 Aug 2026
- 2026Federated Learning for Distributed CNC Tool Wear Prediction11 Aug 2026
- 2026FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning10 Aug 2026
- 2026Capacity Confounds and Coverage Guarantees in Adaptive Sub-model Federated Learning7 Aug 2026
- 2026FL-OA: A Byzantine-Robust Federated Learning Framework with Outsourced Auditing for Intelligent Devices2 Aug 2026
38 more
- 2026FedImp: Enhancing Federated Learning Convergence with Impurity-Based Weighting31 Jul 2026
- 2026F-ACVAE: A Federated Adaptive Conditional Variational Auto-Encoder for Privacy-Preserving Intrusion Detection in IoT Networks6 Jul 2026
- 2026Federated Learning for Object Detection: Enabling Collaborative Drone Learning Without Centralizing Data2 Jul 2026
- 2026Understanding the Robustness of Distributed Self-Supervised Learning Frameworks Against Non-IID Data2 Jul 2026
- 2026C2FL: Clustered Continual Federated Learning under Spatial and Temporal Drift16 Jun 2026
- 2026Distribution Alignment for One-Shot Federated Learning via Optimal Transport15 Jun 2026
- 2026Conflict-Aware Federated Fine-Tuning of Large Language Models with Mixture-of-Experts14 Jun 2026
- 2026From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learning9 Jun 2026
- 2026Learning Emergent Modular Representations in Multi-modality Medical Vision Foundation Models21 May 2026
- 2026FedHPro: Federated Hyper-Prototype Learning via Gradient Matching13 May 2026
- 2026Fed-BAC: Federated Bandit-Guided Additive Clustering in Hierarchical Federated Learning12 May 2026
- 2026Enhancing Federated Quadruplet Learning: Stochastic Client Selection and Embedding Stability Analysis8 May 2026
- 2026FedLLM: A Privacy-Preserving Federated Large Language Model for Explainable Traffic Flow Prediction17 Apr 2026
- 2026Social Hippocampus Memory Learning26 Mar 2026
- 2026Revealing the influence of participant failures on model quality in cross-silo Federated Learning26 Mar 2026
- 2026Aggregation Alignment for Federated Learning with Mixture-of-Experts under Data Heterogeneity22 Mar 2026
- 2026ARMOR: Adaptive Resilience Against Model Poisoning Attacks in Continual Federated Learning for Mobile Indoor Localization20 Mar 2026
- 2026Domain-Skewed Federated Learning with Feature Decoupling and Calibration15 Mar 2026
- 2026FedPrism: Adaptive Personalized Federated Learning under Non-IID Data9 Mar 2026
- 2026Personalized Longitudinal Medical Report Generation via Temporally-Aware Federated Adaptation23 Feb 2026
- 2026PenTiDef: Decentralized Federated Intrusion Detection System with Differential Privacy and Latent-Space Defense via Blockchain Coordination in IIoT20 Feb 2026
- 2026ASFL: An Adaptive Model Splitting and Resource Allocation Framework for Split Federated Learning19 Feb 2026
- 2026Towards Performance-Enhanced Model-Contrastive Federated Learning using Historical Information in Heterogeneous Scenarios12 Feb 2026
- 2026DeepFedNAS: Efficient Hardware-Aware Architecture Adaptation for Heterogeneous IoT Federations via Pareto-Guided Supernet Training21 Jan 2026
- 2026RefProtoFL: Communication-Efficient Federated Learning via External-Referenced Prototype Alignment21 Jan 2026
- 2026Single-Round Clustered Federated Learning via Data Collaboration Analysis for Non-IID Data14 Jan 2026
- 2026Stitch the Fragments: One-Shot Hierarchical Federated Clustering10 Jan 2026
- 2026Federated Learning and Class Imbalances9 Jan 2026
- 2025Zero-Trust Agentic Federated Learning for Secure IIoT Defense Systems29 Dec 2025
- 2025FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computing28 Dec 2025
- 2025DPSR: Differentially Private Sparse Reconstruction via Multi-Stage Denoising for Recommender Systems22 Dec 2025
- 2025Training Together, Diagnosing Better: Federated Learning for Collagen VI-Related Dystrophies18 Dec 2025
- 2025Decentralized Trust for Space AI: Blockchain-Based Federated Learning Across Multi-Vendor LEO Satellite Networks9 Dec 2025
- 2025Delta Sum Learning: an approach for fast and global convergence in Gossip Learning1 Dec 2025
- 2025Closing the Generalization Gap in Parameter-efficient Federated Edge Learning28 Nov 2025
- 2025Privacy Challenges and Solutions in Retrieval-Augmented Generation-Enhanced LLMs for Healthcare Chatbots: A Review of Applications, Risks, and Future Directions14 Nov 2025
- 2025Federated Learning for Pediatric Pneumonia Detection: Enabling Collaborative Diagnosis Without Sharing Patient Data12 Nov 2025
- 2025Data Heterogeneity and Forgotten Labels in Split Federated Learning12 Nov 2025
50 newest of 345 All 345 (JSON)
- 2023Phoenix: A Federated Generative Diffusion Modelcited by 21
- 2024Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Datacited by 10
- 2024FedDM: Enhancing Communication Efficiency and Handling Data Heterogeneity in Federated Diffusion Modelscited by 5
- 2025FedDifRC: Unlocking the Potential of Text-to-Image Diffusion Models in Heterogeneous Federated Learningcited by 3
- 2025Fed-NDIF: A Noise-Embedded Federated Diffusion Model For Low-Count Whole-Body PET Denoisingcited by 1
- 2025FedDifRC: Unlocking the Potential of Text-to-Image Diffusion Models in Heterogeneous Federated Learning9 Jul 2025
- 2025Fed-NDIF: A Noise-Embedded Federated Diffusion Model For Low-Count Whole-Body PET Denoising20 Mar 2025
- 2024FedDM: Enhancing Communication Efficiency and Handling Data Heterogeneity in Federated Diffusion Models20 Jul 2024
- 2024Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Data13 May 2024
- 2023Phoenix: A Federated Generative Diffusion Model7 Jun 2023
- 2026Aggregation Alignment for Federated Learning with Mixture-of-Experts under Data Heterogeneitycited by 4
- 2025FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computingcited by 2
- 2025Towards Efficient Federated Learning of Networked Mixture-of-Experts for Mobile Edge Computingcited by 1
- 2026Conflict-Aware Federated Fine-Tuning of Large Language Models with Mixture-of-Expertscited by 0
- 2026Conflict-Aware Federated Fine-Tuning of Large Language Models with Mixture-of-Experts14 Jun 2026
- 2026Aggregation Alignment for Federated Learning with Mixture-of-Experts under Data Heterogeneity22 Mar 2026
- 2025FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computing28 Dec 2025
- 2025Towards Efficient Federated Learning of Networked Mixture-of-Experts for Mobile Edge Computing3 Nov 2025
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
- 2022NOTE: Robust Continual Test-time Adaptation Against Temporal Correlationcited by 150
- 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
- 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
- 2024Selective Aggregation for Low-Rank Adaptation in Federated Learningcited by 80
- 2023Universal Test-time Adaptation through Weight Ensembling, Diversity Weighting, and Prior Correctioncited by 65
- 2022Client Selection in Federated Learning: Principles, Challenges, and Opportunitiescited by 62
- 2023Better Generative Replay for Continual Federated Learningcited by 53
- 2023Federated Conformal Predictors for Distributed Uncertainty Quantificationcited by 46
- 2022Cross-Silo Federated Learning: Challenges and Opportunitiescited by 38
- 2022Federated Learning for Medical Applications: A Taxonomy, Current Trends, Challenges, and Future Research Directionscited by 34
- 2024An Aggregation-Free Federated Learning for Tackling Data Heterogeneitycited by 33
12 most cited of 330 All 330 (JSON)
- 2026FedMAD: Modulation-Aware Directional Aggregation for Federated Learning in Remote Sensing Image Classification30 Sep 2026
- 2026Making Cross-Continental Federated Learning Repeatable with FLIP: a Multi-Application Study28 Sep 2026
- 2026Federated Deep Clustering Networks for High-Dimensional and Heterogeneous Data18 Sep 2026
- 2026Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation15 Sep 2026
- 2026Decentralized Gossip Learning and Federated Averaging for Histopathology Image Classification15 Sep 2026
- 2026SatDL: Jointly Optimizing Data Redistribution and Training for Satellite-Based Distributed Learning25 Aug 2026
- 2026FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting15 Aug 2026
- 2026Beyond Parameter Space: NTK-Guided Personalized Aggregation for Robust Federated Learning12 Aug 2026
- 2026Federated Learning for Distributed CNC Tool Wear Prediction11 Aug 2026
- 2026FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning10 Aug 2026
- 2026Capacity Confounds and Coverage Guarantees in Adaptive Sub-model Federated Learning7 Aug 2026
- 2026FL-OA: A Byzantine-Robust Federated Learning Framework with Outsourced Auditing for Intelligent Devices2 Aug 2026
12 newest of 330 All 330 (JSON)