# Mitigating Heterogeneity-Induced Drift in Hierarchical Sign-Based Federated Learning

arXiv: https://arxiv.org/abs/2602.02355 · HTML: https://mlsanity.com/paper/2602.02355/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 44 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Builds on (most cited first, 28 of 28)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [signSGD: Compressed Optimisation for Non-Convex Problems](https://mlsanity.com/paper/1802.04434/) (2018)
- [Adaptive Federated Learning in Resource Constrained Edge Computing Systems](https://mlsanity.com/paper/1804.05271/) (2018)
- [FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization](https://mlsanity.com/paper/1909.13014/) (2019)
- [Error Feedback Fixes SignSGD and other Gradient Compression Schemes](https://mlsanity.com/paper/1901.09847/) (2019)
- [Federated Learning via Over-the-Air Computation](https://mlsanity.com/paper/1812.11750/) (2018)
- [Broadband Analog Aggregation for Low-Latency Federated Edge Learning (Extended Version)](https://mlsanity.com/paper/1812.11494/) (2018)
- [On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization](https://mlsanity.com/paper/1905.03817/) (2019)
- [On the Convergence of Local Descent Methods in Federated Learning](https://mlsanity.com/paper/1910.14425/) (2019)
- [The Heavy-Tail Phenomenon in SGD](https://mlsanity.com/paper/2006.04740/) (2020)
- [Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneity](https://mlsanity.com/paper/2012.14453/) (2020)
- [Stochastic Sign Descent Methods: New Algorithms and Better Theory](https://mlsanity.com/paper/1905.12938/) (2019)
- [Hierarchical Federated Learning with Quantization: Convergence Analysis and System Design](https://mlsanity.com/paper/2103.14272/) (2021)
- [On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks](https://mlsanity.com/paper/1912.00018/) (2019)
- [HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Association](https://mlsanity.com/paper/2301.06447/) (2023)
- [Scalable Hierarchical Over-the-Air Federated Learning](https://mlsanity.com/paper/2211.16162/) (2022)
- [Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction](https://mlsanity.com/paper/2406.00489/) (2024)
- [Over-the-Air Federated Edge Learning with Hierarchical Clustering](https://mlsanity.com/paper/2207.09232/) (2022)
- [Adaptive Federated Pruning in Hierarchical Wireless Networks](https://mlsanity.com/paper/2305.09042/) (2023)
- [SignSVRG: fixing SignSGD via variance reduction](https://mlsanity.com/paper/2305.13187/) (2023)
- [Hierarchical Federated Learning with Multi-Timescale Gradient Correction](https://mlsanity.com/paper/2409.18448/) (2024)
- [Hierarchical Over-the-Air Federated Learning with Awareness of Interference and Data Heterogeneity](https://mlsanity.com/paper/2401.01442/) (2024)
- [Device Scheduling and Assignment in Hierarchical Federated Learning for Internet of Things](https://mlsanity.com/paper/2402.02506/) (2024)
- [The Potential of Multilayered Hierarchical Nonterrestrial Networks for 6G: A Comparative Analysis Among Networking Architectures](https://mlsanity.com/paper/2011.08608/) (2020)
- [Hierarchical Federated ADMM](https://mlsanity.com/paper/2409.18796/) (2024)
- [SignSGD with Federated Voting](https://mlsanity.com/paper/2403.16372/) (2024)
- [Accelerating Energy-Efficient Federated Learning in Cell-Free Networks with Adaptive Quantization](https://mlsanity.com/paper/2412.20785/) (2024)

## Abstract (arXiv)

Hierarchical federated learning (HFL) is well suited for large-scale wireless and Internet of Things systems, where devices communicate with nearby edge servers before reaching the cloud. In these environments, uplink bandwidth and latency impose strict communication constraints, making aggressive gradient compression essential. One-bit sign-based stochastic gradient descent methods provide an attractive solution in flat federated settings, but their behavior in hierarchical edge--cloud architectures remains insufficiently understood, especially under inter-cluster data heterogeneity. To address this gap, we develop a sign-based HFL framework in which devices transmit binary stochastic-gradient signs to edge servers, edge servers apply majority voting, and the cloud periodically aggregates edge models. Our analysis reveals that inter-cluster heterogeneity induces a persistent bias term in the convergence bound, reflecting the drift of edge models toward local objectives. This term cannot be removed by increasing the number of training rounds or by tuning standard hyperparameters alone. We therefore propose (mathttDC-HierSignSGD), a drift-corrected sign-based HFL algorithm in which devices apply a cloud-assisted gradient correction before taking the sign. We show that this pre-sign correction mitigates the non-vanishing heterogeneity-induced bias while preserving binary device--edge communication during the repeated local sign-update steps. Experiments under severe inter-cluster heterogeneity demonstrate that (mathttDC-HierSignSGD) improves the stability and accuracy of sign-based HFL and achieves performance comparable to full-precision hierarchical SGD with substantially lower device--edge communication.