# Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learning

arXiv: https://arxiv.org/abs/1911.04655 · HTML: https://mlsanity.com/paper/1911.04655/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2019.

## Built on by (most cited first, 18 of 18)

- [Personalized Federated Learning using Hypernetworks](https://mlsanity.com/paper/2103.04628/) (2021)
- [Distributed Learning in Wireless Networks: Recent Progress and Future Challenges](https://mlsanity.com/paper/2104.02151/) (2021)
- [Personalized Federated Learning with Gaussian Processes](https://mlsanity.com/paper/2106.15482/) (2021)
- [Personalized Federated Learning via Variational Bayesian Inference](https://mlsanity.com/paper/2206.07977/) (2022)
- [Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learning](https://mlsanity.com/paper/2110.08394/) (2021)
- [Communication Efficiency in Federated Learning: Achievements and Challenges](https://mlsanity.com/paper/2107.10996/) (2021)
- [Federated Learning with Heterogeneous Architectures using Graph HyperNetworks](https://mlsanity.com/paper/2201.08459/) (2022)
- [A Communication-Efficient Adaptive Algorithm for Federated Learning under Cumulative Regret](https://mlsanity.com/paper/2301.08869/) (2023)
- [Revisiting Gradient Pruning: A Dual Realization for Defending against Gradient Attacks](https://mlsanity.com/paper/2401.16687/) (2024)
- [Sparse Personalized Federated Learning](https://mlsanity.com/paper/2107.05330/) (2021)
- [Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clustering](https://mlsanity.com/paper/2303.04345/) (2023)
- [Sparse Federated Learning with Hierarchical Personalized Models](https://mlsanity.com/paper/2203.13517/) (2022)
- [Towards Heterogeneous Clients with Elastic Federated Learning](https://mlsanity.com/paper/2106.09433/) (2021)
- [A Personalized Federated Learning Algorithm: an Application in Anomaly Detection](https://mlsanity.com/paper/2111.02627/) (2021)
- [FedMPQ: Secure and Communication-Efficient Federated Learning with Multi-codebook Product Quantization](https://mlsanity.com/paper/2404.13575/) (2024)
- [FedFT: Improving Communication Performance for Federated Learning with Frequency Space Transformation](https://mlsanity.com/paper/2409.05242/) (2024)
- [RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image Interpretation](https://mlsanity.com/paper/2312.17530/) (2023)
- [Quantization in Federated Learning: Methods, Challenges and Future Directions](https://mlsanity.com/paper/2606.26822/) (2026)

## Abstract (arXiv)

The high cost of communicating gradients is a major bottleneck for federated learning, as the bandwidth of the participating user devices is limited. Existing gradient compression algorithms are mainly designed for data centers with high-speed network and achieve O(√d log d) per-iteration communication cost at best, where d is the size of the model. We propose hyper-sphere quantization (HSQ), a general framework that can be configured to achieve a continuum of trade-offs between communication efficiency and gradient accuracy. In particular, at the high compression ratio end, HSQ provides a low per-iteration communication cost of O(log d), which is favorable for federated learning. We prove the convergence of HSQ theoretically and show by experiments that HSQ significantly reduces the communication cost of model training without hurting convergence accuracy.