mlsanityData to 14 Oct 2026

FedSym: Unleashing the Power of Entropy for Benchmarking the Algorithms for Federated Learning

arXiv 2310.07807 ↗First posted 11 Oct 2023cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is a decentralized machine learning approach where independent learners process data privately. Its goal is to create a robust and accurate model by aggregating and retraining local models over multiple rounds. However, FL faces challenges regarding data heterogeneity and model aggregation effectiveness. In order to simulate real-world data, researchers use methods for data partitioning that transform a dataset designated for centralized learning into a group of sub-datasets suitable for distributed machine learning with different data heterogeneity. In this paper, we study the currently popular data partitioning techniques and visualize their main disadvantages: the lack of precision in the data diversity, which leads to unreliable heterogeneity indexes, and the inability to incrementally challenge the FL algorithms. To resolve this problem, we propose a method that leverages entropy and symmetry to construct 'the most challenging' and controllable data distributions with gradual difficulty. We introduce a metric to measure data heterogeneity among the learning agents and a transformation technique that divides any dataset into splits with precise data diversity. Through a comparative study, we demonstrate the superiority of our method over existing FL data partitioning approaches, showcasing its potential to challenge model aggregation algorithms. Experimental results indicate that our approach gradually challenges the FL strategies, and the models trained on FedSym distributions are more distinct.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 54 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

29 of 54 references.

References · 29

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2019Advances and Open Problems in Federated Learningcited by 3,046
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  5. 2018Federated Learning with Non-IID Datacited by 1,108
  6. 2020Adaptive Federated Optimizationcited by 1,046
  7. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  8. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  9. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  10. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  11. 2020Federated Learning with Matched Averagingcited by 564
  12. 2021Federated Learning Based on Dynamic Regularizationcited by 562
17 more
  1. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 459
  2. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  3. 2021Federated Learning on Non-IID Data: A Surveycited by 346
  4. 2019Robust Aggregation for Federated Learningcited by 317
  5. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 283
  6. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  7. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  8. 2020The Discrete Gaussian for Differential Privacycited by 135
  9. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 113
  10. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  11. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 109
  12. 2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
  13. 2020Practical One-Shot Federated Learning for Cross-Silo Settingcited by 77
  14. 2020LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasetscited by 60
  15. 2022On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Datacited by 38
  16. 2022Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarkscited by 24
  17. 2021FedRAD: Federated Robust Adaptive Distillationcited by 11

Built on by

Not cited yet.