# High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learning

arXiv: https://arxiv.org/abs/1910.03865 · HTML: https://mlsanity.com/paper/1910.03865/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2019.

## Built on by (most cited first, 30 of 32)

- [Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications](https://mlsanity.com/paper/2111.12444/) (2021)
- [Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devices](https://mlsanity.com/paper/2109.15258/) (2021)
- [Hierarchical Federated Learning with Quantization: Convergence Analysis and System Design](https://mlsanity.com/paper/2103.14272/) (2021)
- [Dynamic Scheduling for Over-the-Air Federated Edge Learning with Energy Constraints](https://mlsanity.com/paper/2106.00490/) (2021)
- [Communication-Efficient Federated Learning via Quantized Compressed Sensing](https://mlsanity.com/paper/2111.15071/) (2021)
- [Communication-Efficient Split Learning via Adaptive Feature-Wise Compression](https://mlsanity.com/paper/2307.10805/) (2023)
- [Edge Federated Learning Via Unit-Modulus Over-The-Air Computation](https://mlsanity.com/paper/2101.12051/) (2021)
- [On the Convergence of Decentralized Federated Learning Under Imperfect Information Sharing](https://mlsanity.com/paper/2303.10695/) (2023)
- [Turning Channel Noise into an Accelerator for Over-the-Air Principal Component Analysis](https://mlsanity.com/paper/2104.10095/) (2021)
- [Wirelessly Powered Federated Edge Learning: Optimal Tradeoffs Between Convergence and Power Transfer](https://mlsanity.com/paper/2102.12357/) (2021)
- [Federated Learning over Noisy Channels: Convergence Analysis and Design Examples](https://mlsanity.com/paper/2101.02198/) (2021)
- [Accelerating Federated Edge Learning via Topology Optimization](https://mlsanity.com/paper/2204.00489/) (2022)
- [Enabling All In-Edge Deep Learning: A Literature Review](https://mlsanity.com/paper/2204.03326/) (2022)
- [Riemannian Low-Rank Model Compression for Federated Learning with Over-the-Air Aggregation](https://mlsanity.com/paper/2306.02433/) (2023)
- [Improved Convergence Analysis and SNR Control Strategies for Federated Learning in the Presence of Noise](https://mlsanity.com/paper/2307.07406/) (2023)
- [Distributed Learning based on 1-Bit Gradient Coding in the Presence of Stragglers](https://mlsanity.com/paper/2403.14716/) (2024)
- [Efficient Model Compression for Hierarchical Federated Learning](https://mlsanity.com/paper/2405.17522/) (2024)
- [Communication-efficient Vertical Federated Learning via Compressed Error Feedback](https://mlsanity.com/paper/2406.14420/) (2024)
- [Dynamic Scheduling for Vehicle-to-Vehicle Communications Enhanced Federated Learning](https://mlsanity.com/paper/2406.17470/) (2024)
- [Energy-Efficient Edge Learning via Joint Data Deepening-and-Prefetching](https://mlsanity.com/paper/2402.11925/) (2024)
- [FedNMUT -- Federated Noisy Model Update Tracking Convergence Analysis](https://mlsanity.com/paper/2403.13247/) (2024)
- [TAGC: Optimizing Gradient Communication in Distributed Transformer Training](https://mlsanity.com/paper/2504.05638/) (2025)
- [Smoothed Differential Privacy](https://mlsanity.com/paper/2107.01559/) (2021)
- [Empirical Analysis on Top-k Gradient Sparsification for Distributed Deep Learning in a Supercomputing Environment](https://mlsanity.com/paper/2209.08497/) (2022)
- [Accelerating Wireless Federated Learning via Nesterov's Momentum and Distributed Principle Component Analysis](https://mlsanity.com/paper/2303.17885/) (2023)
- [GQFedWAvg: Optimization-Based Quantized Federated Learning in General Edge Computing Systems](https://mlsanity.com/paper/2306.07497/) (2023)
- [Federated Learning-based Collaborative Wideband Spectrum Sensing and Scheduling for UAVs in UTM Systems](https://mlsanity.com/paper/2406.01727/) (2024)
- [Unit-Modulus Wireless Federated Learning Via Penalty Alternating Minimization](https://mlsanity.com/paper/2108.13669/) (2021)
- [Random Orthogonalization for Federated Learning in Massive MIMO Systems](https://mlsanity.com/paper/2210.09881/) (2022)
- [Efficient Wireless Federated Learning via Low-Rank Gradient Factorization](https://mlsanity.com/paper/2401.07496/) (2024)

## Abstract (arXiv)

Edge machine learning involves the deployment of learning algorithms at the wireless network edge so as to leverage massive mobile data for enabling intelligent applications. The mainstream edge learning approach, federated learning, has been developed based on distributed gradient descent. Based on the approach, stochastic gradients are computed at edge devices and then transmitted to an edge server for updating a global AI model. Since each stochastic gradient is typically high-dimensional (with millions to billions of coefficients), communication overhead becomes a bottleneck for edge learning. To address this issue, we propose in this work a novel framework of hierarchical stochastic gradient quantization and study its effect on the learning performance. First, the framework features a practical hierarchical architecture for decomposing the stochastic gradient into its norm and normalized block gradients, and efficiently quantizes them using a uniform quantizer and a low-dimensional codebook on a Grassmann manifold, respectively. Subsequently, the quantized normalized block gradients are scaled and cascaded to yield the quantized normalized stochastic gradient using a so-called hinge vector designed under the criterion of minimum distortion. The hinge vector is also efficiently compressed using another low-dimensional Grassmannian quantizer. The other feature of the framework is a bit-allocation scheme for reducing the quantization error. The scheme determines the resolutions of the low-dimensional quantizers in the proposed framework. The framework is proved to guarantee model convergency by analyzing the convergence rate as a function of the quantization bits. Furthermore, by simulation, our design is shown to substantially reduce the communication overhead compared with the state-of-the-art signSGD scheme, while both achieve similar learning accuracies.