Heterogeneity-Aware Client Sampling for Optimal and Efficient Federated Learning
Not on the map. Why?
Abstract (arXiv)
Federated learning (FL) commonly involves clients with diverse communication and computational capabilities. Such heterogeneity can significantly distort the optimization dynamics and lead to objective inconsistency, where the global model converges to an incorrect stationary point potentially far from the pursued optimum. Despite its critical impact, the joint effect of communication and computation heterogeneity has remained largely unexplored, due to the intrinsic complexity of their interaction. In this paper, we reveal the fundamentally distinct mechanisms through which heterogeneous communication and computation drive inconsistency in FL. To the best of our knowledge, this is the first unified theoretical analysis of general heterogeneous FL, offering a principled understanding of how these two forms of heterogeneity jointly distort the optimization trajectory under arbitrary choices of local solvers. Motivated by these insights, we propose Federated Heterogeneity-Aware Client Sampling, FedACS, a universal method to eliminate all types of objective inconsistency. We theoretically prove that FedACS converges to the correct optimum at a rate of O(1/√R), even in dynamic heterogeneous environments. Extensive experiments across multiple datasets show that FedACS outperforms state-of-the-art and category-specific baselines by 4.3%-36%, while reducing communication costs by 22%-89% and computation loads by 14%-105%, respectively.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 41 references that are in the map. First posted on arXiv 16 May 2025.
Builds on
24 of 41 references.
References · 24
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
- 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
- 2021Federated Learning on Non-IID Data: A Surveycited by 346
- 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
- 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 175
- 2020Optimal Client Sampling for Federated Learningcited by 124
- 2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
- 2020Over-the-Air Federated Learning from Heterogeneous Datacited by 73
- 2021Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Samplingcited by 66
- 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 64
- 2020Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneitycited by 56
12 more
- 2021Quantized Federated Learning under Transmission Delay and Outage Constraintscited by 37
- 2018New Convergence Aspects of Stochastic Gradient Algorithmscited by 30
- 2020Energy-Efficient Federated Edge Learning with Joint Communication and Computation Designcited by 30
- 2020Federated Learning in Unreliable and Resource-Constrained Cellular Wireless Networkscited by 29
- 2022Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimizationcited by 28
- 2022Data Heterogeneity-Robust Federated Learning via Group Client Selection in Industrial IoTcited by 25
- 2022DELTA: Diverse Client Sampling for Fasting Federated Learningcited by 24
- 2022Wireless Quantized Federated Learning: A Joint Computation and Communication Designcited by 15
- 2023Solving a Class of Non-Convex Minimax Optimization in Federated Learningcited by 10
- 2024Efficient Federated Learning against Heterogeneous and Non-stationary Client Unavailabilitycited by 9
- 2023Blind Federated Learning via Over-the-Air q-QAMcited by 8
- 2024Privacy Preserving Semi-Decentralized Mean Estimation over Intermittently-Connected Networkscited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.