mlsanityData to 7 Oct 2026

Adaptive Federated Learning via New Entropy Approach

arXiv 2303.14966 ↗First posted 27 Mar 2023cs.DC cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) has emerged as a prominent distributed machine learning framework that enables geographically discrete clients to train a global model collaboratively while preserving their privacy-sensitive data. However, due to the non-independent-and-identically-distributed (Non-IID) data generated by heterogeneous clients, the performances of the conventional federated optimization schemes such as FedAvg and its variants deteriorate, requiring the design to adaptively adjust specific model parameters to alleviate the negative influence of heterogeneity. In this paper, by leveraging entropy as a new metric for assessing the degree of system disorder, we propose an adaptive FEDerated learning algorithm based on ENTropy theory (FedEnt) to alleviate the parameter deviation among heterogeneous clients and achieve fast convergence. Nevertheless, given the data disparity and parameter deviation of heterogeneous clients, determining the optimal dynamic learning rate for each client becomes a challenging task as there is no communication among participating clients during the local training epochs. To enable a decentralized learning rate for each participating client, we first introduce the mean-field terms to estimate the components associated with other clients' local parameters. Furthermore, we provide rigorous theoretical analysis on the existence and determination of the mean-field estimators. Based on the mean-field estimators, the closed-form adaptive learning rate for each client is derived by constructing the Hamilton equation. Moreover, the convergence rate of our proposed FedEnt is proved. The extensive experimental results on the real-world datasets (i.e., MNIST, EMNIST-L, CIFAR10, and CIFAR100) show that our FedEnt algorithm surpasses FedAvg and its variants (i.e., FedAdam, FedProx, and FedDyn) under Non-IID settings and achieves a faster convergence rate.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 28 of its 64 references that are in the map. First posted on arXiv 27 Mar 2023.

Builds on

28 of 64 references.

References · 28

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2018Federated Learning with Non-IID Datacited by 1,099
  5. 2020Adaptive Federated Optimizationcited by 1,036
  6. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  7. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  8. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  9. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  10. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  11. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  12. 2021Towards Personalized Federated Learningcited by 462
16 more
  1. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  2. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  3. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
  4. 2019Asynchronous Federated Optimizationcited by 264
  5. 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
  6. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  7. 2018AdaGrad stepsizes: Sharp convergence over nonconvex landscapescited by 182
  8. 2018Understanding the impact of entropy on policy optimizationcited by 171
  9. 2019Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregationcited by 99
  10. 2020Fast-Convergent Federated Learning with Adaptive Weightingcited by 67
  11. 2022Client Selection in Federated Learning: Principles, Challenges, and Opportunitiescited by 62
  12. 2020Fast-Convergent Federated Learningcited by 61
  13. 2019Accelerating DNN Training in Wireless Federated Edge Learning Systemscited by 34
  14. 2020Adaptive Federated Learning and Digital Twin for Industrial Internet of Thingscited by 27
  15. 2021Communication-efficient SGD: From Local SGD to One-Shot Averagingcited by 19
  16. 2021Genetic CFL: Optimization of Hyper-Parameters in Clustered Federated Learningcited by 6

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2024Optimizing Federated Learning by Entropy-Based Client Selectioncited by 3
  2. 2025Towards Heterogeneity-Aware and Energy-Efficient Topology Optimization for Decentralized Federated Learning in Edge Environmentcited by 3
  3. 2025FedShard: Federated Unlearning with Efficiency Fairness and Performance Fairnesscited by 0