mlsanityData to 7 Oct 2026

Mitigating Heterogeneity-Induced Drift in Hierarchical Sign-Based Federated Learning

arXiv 2602.02355 ↗First posted 2 Feb 2026cs.DC cs.IT cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Hierarchical federated learning (HFL) is well suited for large-scale wireless and Internet of Things systems, where devices communicate with nearby edge servers before reaching the cloud. In these environments, uplink bandwidth and latency impose strict communication constraints, making aggressive gradient compression essential. One-bit sign-based stochastic gradient descent methods provide an attractive solution in flat federated settings, but their behavior in hierarchical edge--cloud architectures remains insufficiently understood, especially under inter-cluster data heterogeneity. To address this gap, we develop a sign-based HFL framework in which devices transmit binary stochastic-gradient signs to edge servers, edge servers apply majority voting, and the cloud periodically aggregates edge models. Our analysis reveals that inter-cluster heterogeneity induces a persistent bias term in the convergence bound, reflecting the drift of edge models toward local objectives. This term cannot be removed by increasing the number of training rounds or by tuning standard hyperparameters alone. We therefore propose (mathttDC-HierSignSGD), a drift-corrected sign-based HFL algorithm in which devices apply a cloud-assisted gradient correction before taking the sign. We show that this pre-sign correction mitigates the non-vanishing heterogeneity-induced bias while preserving binary device--edge communication during the repeated local sign-update steps. Experiments under severe inter-cluster heterogeneity demonstrate that (mathttDC-HierSignSGD) improves the stability and accuracy of sign-based HFL and achieves performance comparable to full-precision hierarchical SGD with substantially lower device--edge communication.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 44 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

28 of 44 references.

References · 28

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Towards Federated Learning at Scale: System Designcited by 826
  3. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  4. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  5. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  6. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 306
  7. 2018Federated Learning via Over-the-Air Computationcited by 215
  8. 2018Broadband Analog Aggregation for Low-Latency Federated Edge Learning (Extended Version)cited by 186
  9. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 178
  10. 2019On the Convergence of Local Descent Methods in Federated Learningcited by 150
  11. 2020The Heavy-Tail Phenomenon in SGDcited by 122
  12. 2020Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneitycited by 56
16 more
  1. 2019Stochastic Sign Descent Methods: New Algorithms and Better Theorycited by 49
  2. 2021Hierarchical Federated Learning with Quantization: Convergence Analysis and System Designcited by 47
  3. 2019On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networkscited by 42
  4. 2023HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Associationcited by 17
  5. 2022Scalable Hierarchical Over-the-Air Federated Learningcited by 14
  6. 2024Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reductioncited by 13
  7. 2022Over-the-Air Federated Edge Learning with Hierarchical Clusteringcited by 10
  8. 2023Adaptive Federated Pruning in Hierarchical Wireless Networkscited by 10
  9. 2023SignSVRG: fixing SignSGD via variance reductioncited by 7
  10. 2024Hierarchical Federated Learning with Multi-Timescale Gradient Correctioncited by 7
  11. 2024Hierarchical Over-the-Air Federated Learning with Awareness of Interference and Data Heterogeneitycited by 6
  12. 2024Device Scheduling and Assignment in Hierarchical Federated Learning for Internet of Thingscited by 6
  13. 2020The Potential of Multilayered Hierarchical Nonterrestrial Networks for 6G: A Comparative Analysis Among Networking Architecturescited by 4
  14. 2024Hierarchical Federated ADMMcited by 4
  15. 2024SignSGD with Federated Votingcited by 2
  16. 2024Accelerating Energy-Efficient Federated Learning in Cell-Free Networks with Adaptive Quantizationcited by 1

Built on by

Not cited yet.