mlsanityData to 7 Oct 2026

Benchmarking Data Heterogeneity Evaluation Approaches for Personalized Federated Learning

arXiv 2410.07286 ↗First posted 9 Oct 2024cs.LG cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

There is growing research interest in measuring the statistical heterogeneity of clients' local datasets. Such measurements are used to estimate the suitability for collaborative training of personalized federated learning (PFL) models. Currently, these research endeavors are taking place in silos and there is a lack of a unified benchmark to provide a fair and convenient comparison among various approaches in common settings. We aim to bridge this important gap in this paper. The proposed benchmarking framework currently includes six representative approaches. Extensive experiments have been conducted to compare these approaches under five standard non-IID FL settings, providing much needed insights into which approaches are advantageous under which settings. The proposed framework offers useful guidance on the suitability of various data divergence measures in FL systems. It is beneficial for keeping related research activities on the right track in terms of: (1) designing PFL schemes, (2) selecting appropriate data heterogeneity evaluation approaches for specific FL application scenarios, and (3) addressing fairness issues in collaborative model training. The code is available at https://github.com/Xiaoni-61/DH-Benchmark.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 33 of its 66 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

33 of 66 references.

References · 33

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,352
  6. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  7. 2018LEAF: A Benchmark for Federated Settingscited by 780
  8. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 617
  9. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  10. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  11. 2020Federated Learning with Matched Averagingcited by 564
  12. 2021Towards Personalized Federated Learningcited by 462
21 more
  1. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  2. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  3. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  4. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  5. 2021Personalized Federated Learning using Hypernetworkscited by 210
  6. 2020Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challengescited by 138
  7. 2020Learning the Pareto Front with Hypernetworkscited by 122
  8. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 112
  9. 2020Collaborative Fairness in Federated Learningcited by 95
  10. 2021GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learningcited by 73
  11. 2020Model-sharing Games: Analyzing Federated Learning Under Voluntary Participationcited by 60
  12. 2020Federated Learning with Only Positive Labelscited by 55
  13. 2021Incentive Mechanisms for Federated Learning: From Economic and Game Theoretic Perspectivecited by 47
  14. 2023Optimizing the Collaboration Structure in Cross-Silo Federated Learningcited by 19
  15. 2021Collaboration Equilibrium in Federated Learningcited by 15
  16. 2021MarS-FL: Enabling Competitors to Collaborate in Federated Learningcited by 11
  17. 2020Continuous Transfer Learning with Label-informed Distribution Alignmentcited by 6
  18. 2023FedCompetitors: Harmonious Collaboration in Federated Learning with Competing Participantscited by 4
  19. 2024Measuring Data Similarity for Efficient Federated Learning: A Feasibility Studycited by 4
  20. 2022Efficient Medical Image Assessment via Self-supervised Learningcited by 2
  21. 2024LEAP: Optimization Hierarchical Federated Learning on Non-IID Data with Coalition Formation Gamecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024Free-Rider and Conflict Aware Collaboration Formation for Cross-Silo Federated Learningcited by 9
  2. 2025Voronoi-grid-based Pareto Front Learning and Its Application to Collaborative Federated Learningcited by 0
  3. 2026FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environmentscited by 0