mlsanityData to 14 Oct 2026

HeteRo-Select: Informativeness as the Participation Driver in Heterogeneous Federated Learning

arXiv 2508.06692 ↗First posted 8 Aug 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated learning systems typically allocate gradient compression by link speed. This is sensible when bandwidth and data informativeness align. However, under non-IID data, these signals often decorrelate or invert. A bandwidth-driven allocator then risks compressing the most informative gradients hardest. We propose HeteRo-Select, a framework that replaces bandwidth with a per-client informativeness score as the primary driver of compression. The score jointly governs three decisions per round: client selection, compression ratio, and server aggregation weight, with bandwidth retained only as a hard ceiling. Score-proportional selection provably reduces the effective heterogeneity of the chosen subset; score-proportional compression provably lowers aggregate top-k error at fixed traffic. Under the exact FedCG simulation protocol, HeteRo-Select delivers a 1.78× speedup and an 18.2% reduction in traffic on CIFAR-10. The same configuration, unchanged, scales from a 7,850-parameter logistic regression to an 11.27M-parameter ResNet-18, hitting the accuracy target on three of four benchmarks. When bandwidth and informativeness are deliberately anti-correlated, the method still achieves the target accuracy with less traffic than the normal-bandwidth run.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 42 references that are in the map. First posted on arXiv 8 Aug 2025.

Builds on

34 of 42 references.

References · 34

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  6. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  7. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  8. 2020Adaptive Federated Optimizationcited by 1,046
  9. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  10. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 571
  11. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  12. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
22 more
  1. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 478
  2. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
  3. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  4. 2018Sparsified SGD with Memorycited by 376
  5. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
  6. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 352
  7. 2019PyHessian: Neural Networks Through the Lens of the Hessiancited by 292
  8. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 256
  9. 2017Sparse Communication for Distributed Gradient Descentcited by 249
  10. 2017Gradient Sparsification for Communication-Efficient Distributed Optimizationcited by 201
  11. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 198
  12. 2021EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedbackcited by 133
  13. 2020Optimal Client Sampling for Federated Learningcited by 124
  14. 2021Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Samplingcited by 66
  15. 2020An Efficiency-boosting Client Selection Scheme for Federated Learning with Fairness Guaranteecited by 62
  16. 2021FedCor: Correlation-Based Active Client Selection Strategy for Heterogeneous Federated Learningcited by 43
  17. 2015Coordinate Descent Converges Faster with the Gauss-Southwell Rule Than Random Selectioncited by 39
  18. 2022DELTA: Diverse Client Sampling for Fasting Federated Learningcited by 24
  19. 2023Fairness-Aware Client Selection for Federated Learningcited by 15
  20. 2023FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learningcited by 14
  21. 2024FedSTaS: Client Stratification and Client Level Sampling for Efficient Federated Learningcited by 2
  22. 2024Emulating Full Participation: An Effective and Fair Client Selection Strategy for Federated Learningcited by 1

Built on by

Not cited yet.