FedEntropy: Efficient Device Grouping for Federated Learning Using Maximum Entropy Judgment
Not on the map. Why?
Abstract (arXiv)
Along with the popularity of Artificial Intelligence (AI) and Internet-of-Things (IoT), Federated Learning (FL) has attracted steadily increasing attentions as a promising distributed machine learning paradigm, which enables the training of a central model on for numerous decentralized devices without exposing their privacy. However, due to the biased data distributions on involved devices, FL inherently suffers from low classification accuracy in non-IID scenarios. Although various device grouping method have been proposed to address this problem, most of them neglect both i) distinct data distribution characteristics of heterogeneous devices, and ii) contributions and hazards of local models, which are extremely important in determining the quality of global model aggregation. In this paper, we present an effective FL method named FedEntropy with a novel dynamic device grouping scheme, which makes full use of the above two factors based on our proposed maximum entropy judgement heuristic.Unlike existing FL methods that directly aggregate local models returned from all the selected devices, in one FL round FedEntropy firstly makes a judgement based on the pre-collected soft labels of selected devices and then only aggregates the local models that can maximize the overall entropy of these soft labels. Without collecting local models that are harmful for aggregation, FedEntropy can effectively improve global model accuracy while reducing the overall communication overhead. Comprehensive experimental results on well-known benchmarks show that, FedEntropy not only outperforms state-of-the-art FL methods in terms of model accuracy and communication overhead, but also can be integrated into them to enhance their classification performance.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 23 of its 33 references that are in the map. First posted on arXiv 24 May 2022.
Builds on
23 of 33 references.
References · 23
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2018Federated Optimization in Heterogeneous Networkscited by 2,987
- 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
- 2021Model-Contrastive Federated Learningcited by 766
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
- 2021Federated Learning Based on Dynamic Regularizationcited by 562
- 2020An Efficient Framework for Clustered Federated Learningcited by 511
- 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 464
- 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
- 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
- 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
- 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
11 more
- 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
- 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 246
- 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
- 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
- 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 123
- 2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
- 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 68
- 2018Maximum-Entropy Fine-Grained Classificationcited by 65
- 2020Performance Optimization for Federated Person Re-identification via Benchmark Analysiscited by 50
- 2021Node Selection Toward Faster Convergence for Federated Learning on Non-IID Datacited by 39
- 2020FedCluster: Boosting the Convergence of Federated Learning via Cluster-Cyclingcited by 27
Built on by
Citing papers per year: 2023: 2; 2024: 1.
3 papers.