Enhancing Federated Learning Convergence with Dynamic Data Queue and Data Entropy-driven Participant Selection
Not on the map. Why?
Abstract (arXiv)
Federated Learning (FL) is a decentralized approach for collaborative model training on edge devices. This distributed method of model training offers advantages in privacy, security, regulatory compliance, and cost-efficiency. Our emphasis in this research lies in addressing statistical complexity in FL, especially when the data stored locally across devices is not identically and independently distributed (non-IID). We have observed an accuracy reduction of up to approximately 10% to 30%, particularly in skewed scenarios where each edge device trains with only 1 class of data. This reduction is attributed to weight divergence, quantified using the Euclidean distance between device-level class distributions and the population distribution, resulting in a bias term ((δk)). As a solution, we present a method to improve convergence in FL by creating a global subset of data on the server and dynamically distributing it across devices using a Dynamic Data queue-driven Federated Learning (DDFL). Next, we leverage Data Entropy metrics to observe the process during each training round and enable reasonable device selection for aggregation. Furthermore, we provide a convergence analysis of our proposed DDFL to justify their viability in practical FL scenarios, aiming for better device selection, a non-sub-optimal global model, and faster convergence. We observe that our approach results in a substantial accuracy boost of approximately 5% for the MNIST dataset, around 18% for CIFAR-10, and 20% for CIFAR-100 with a 10% global subset of data, outperforming the state-of-the-art (SOTA) aggregation algorithms.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 41 references that are in the map. First posted on arXiv 23 Oct 2024.
Builds on
23 of 41 references.
References · 23
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
- 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
- 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
- 2018Federated Learning with Non-IID Datacited by 1,099
- 2017Federated Multi-Task Learningcited by 669
- 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
- 2018Federated Learning for Mobile Keyboard Predictioncited by 565
- 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
- 2019Fair Resource Allocation in Federated Learningcited by 389
11 more
- 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
- 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
- 2019A Joint Learning and Communications Framework for Federated Learning over Wireless Networkscited by 262
- 2018Don't Use Large Mini-Batches, Use Local SGDcited by 201
- 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 193
- 2020Privacy and Robustness in Federated Learning: Attacks and Defensescited by 163
- 2020Privacy-preserving Traffic Flow Prediction: A Federated Learning Approachcited by 82
- 2020An Efficiency-boosting Client Selection Scheme for Federated Learning with Fairness Guaranteecited by 61
- 2020Decentralized Federated Learning via Mutual Knowledge Transfercited by 42
- 2022Gradient and Channel Aware Dynamic Scheduling for Over-the-Air Computation in Federated Edge Learning Systemscited by 20
- 2022Warmup and Transfer Knowledge-Based Federated Learning Approach for IoT Continuous Authenticationcited by 3
Built on by
Not cited yet.