mlsanityData to 7 Oct 2026

Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification

arXiv 1909.06335 ↗First posted 13 Sep 2019cs.LG cs.CV stat.MLcited by 836

Not on the map. Why?

Abstract (arXiv)

Federated Learning enables visual models to be trained in a privacy-preserving way using real-world data from mobile devices. Given their distributed nature, the statistics of the data across these devices is likely to differ significantly. In this work, we look at the effect such non-identical data distributions has on visual classification via Federated Learning. We propose a way to synthesize datasets with a continuous range of identicalness and provide performance measures for the Federated Averaging algorithm. We show that performance degrades as distributions differ more, and propose a mitigation strategy via server momentum. Experiments on CIFAR-10 demonstrate improved classification performance over a range of non-identicalness, with classification accuracy improved from 30.1% to 76.9% in the most skewed settings.

Key facts. Cited by 836 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 96; 2022: 132; 2023: 164; 2024: 177; 2025: 155; 2026: 111.

836 papers.

Sort:

By area:

  1. 2021Model-Contrastive Federated Learningcited by 762
  2. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  3. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  4. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  5. 2021Towards Personalized Federated Learningcited by 462
  6. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  7. 2021A Field Guide to Federated Optimizationcited by 312
  8. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
  9. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  10. 2021Personalized Federated Learning using Hypernetworkscited by 210
  11. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  12. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
38 more
  1. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  2. 2022NOTE: Robust Continual Test-time Adaptation Against Temporal Correlationcited by 150
  3. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  4. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 124
  5. 2022FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settingscited by 121
  6. 2021Decentralized Federated Averagingcited by 114
  7. 2021FairFed: Enabling Group Fairness in Federated Learningcited by 114
  8. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 112
  9. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  10. 2022Improving Generalization in Federated Learning by Seeking Flat Minimacited by 98
  11. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
  12. 2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
  13. 2023Deep Model Fusion: A Surveycited by 89
  14. 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
  15. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 86
  16. 2023Multimodal Federated Learning via Contrastive Representation Ensemblecited by 85
  17. 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
  18. 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 80
  19. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  20. 2021Personalized Federated Learning with Gaussian Processescited by 76
  21. 2021Differentially Private Federated Learning on Heterogeneous Datacited by 74
  22. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  23. 2021On Large-Cohort Training for Federated Learningcited by 73
  24. 2021Fast Federated Learning in the Presence of Arbitrary Device Unavailabilitycited by 72
  25. 2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
  26. 2022Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learningcited by 71
  27. 2021Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Datacited by 69
  28. 2023FedExP: Speeding Up Federated Averaging via Extrapolationcited by 68
  29. 2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 67
  30. 2023FedDisco: Federated Learning with Discrepancy-Aware Collaborationcited by 66
  31. 2022FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learningcited by 65
  32. 2023Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneitycited by 64
  33. 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
  34. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  35. 2022FedCorr: Multi-Stage Federated Learning for Label Noise Correctioncited by 61
  36. 2022FedNest: Federated Bilevel, Minimax, and Compositional Optimizationcited by 60
  37. 2022Communication-Efficient Adaptive Federated Learningcited by 55
  38. 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 54

50 most cited of 836 All 836 (JSON)