mlsanityData to 14 Oct 2026

High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learning

arXiv 1910.03865 ↗First posted 9 Oct 2019cs.IT cs.LG eess.SPcited by 32

Not on the map. Why?

Abstract (arXiv)

Edge machine learning involves the deployment of learning algorithms at the wireless network edge so as to leverage massive mobile data for enabling intelligent applications. The mainstream edge learning approach, federated learning, has been developed based on distributed gradient descent. Based on the approach, stochastic gradients are computed at edge devices and then transmitted to an edge server for updating a global AI model. Since each stochastic gradient is typically high-dimensional (with millions to billions of coefficients), communication overhead becomes a bottleneck for edge learning. To address this issue, we propose in this work a novel framework of hierarchical stochastic gradient quantization and study its effect on the learning performance. First, the framework features a practical hierarchical architecture for decomposing the stochastic gradient into its norm and normalized block gradients, and efficiently quantizes them using a uniform quantizer and a low-dimensional codebook on a Grassmann manifold, respectively. Subsequently, the quantized normalized block gradients are scaled and cascaded to yield the quantized normalized stochastic gradient using a so-called hinge vector designed under the criterion of minimum distortion. The hinge vector is also efficiently compressed using another low-dimensional Grassmannian quantizer. The other feature of the framework is a bit-allocation scheme for reducing the quantization error. The scheme determines the resolutions of the low-dimensional quantizers in the proposed framework. The framework is proved to guarantee model convergency by analyzing the convergence rate as a function of the quantization bits. Furthermore, by simulation, our design is shown to substantially reduce the communication overhead compared with the state-of-the-art signSGD scheme, while both achieve similar learning accuracies.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 11; 2022: 4; 2023: 6; 2024: 9; 2025: 2.

32 papers.

Sort:

  1. 2021Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applicationscited by 97
  2. 2021Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devicescited by 56
  3. 2021Hierarchical Federated Learning with Quantization: Convergence Analysis and System Designcited by 47
  4. 2021Dynamic Scheduling for Over-the-Air Federated Edge Learning with Energy Constraintscited by 36
  5. 2021Communication-Efficient Federated Learning via Quantized Compressed Sensingcited by 18
  6. 2023Communication-Efficient Split Learning via Adaptive Feature-Wise Compressioncited by 18
  7. 2021Edge Federated Learning Via Unit-Modulus Over-The-Air Computationcited by 16
  8. 2023On the Convergence of Decentralized Federated Learning Under Imperfect Information Sharingcited by 16
  9. 2021Turning Channel Noise into an Accelerator for Over-the-Air Principal Component Analysiscited by 14
  10. 2021Wirelessly Powered Federated Edge Learning: Optimal Tradeoffs Between Convergence and Power Transfercited by 10
  11. 2021Federated Learning over Noisy Channels: Convergence Analysis and Design Examplescited by 8
  12. 2022Accelerating Federated Edge Learning via Topology Optimizationcited by 4
20 more
  1. 2022Enabling All In-Edge Deep Learning: A Literature Reviewcited by 4
  2. 2023Riemannian Low-Rank Model Compression for Federated Learning with Over-the-Air Aggregationcited by 4
  3. 2023Improved Convergence Analysis and SNR Control Strategies for Federated Learning in the Presence of Noisecited by 4
  4. 2024Distributed Learning based on 1-Bit Gradient Coding in the Presence of Stragglerscited by 4
  5. 2024Efficient Model Compression for Hierarchical Federated Learningcited by 3
  6. 2024Communication-efficient Vertical Federated Learning via Compressed Error Feedbackcited by 3
  7. 2024Dynamic Scheduling for Vehicle-to-Vehicle Communications Enhanced Federated Learningcited by 3
  8. 2024Energy-Efficient Edge Learning via Joint Data Deepening-and-Prefetchingcited by 2
  9. 2024FedNMUT -- Federated Noisy Model Update Tracking Convergence Analysiscited by 2
  10. 2025TAGC: Optimizing Gradient Communication in Distributed Transformer Trainingcited by 2
  11. 2021Smoothed Differential Privacycited by 1
  12. 2022Empirical Analysis on Top-k Gradient Sparsification for Distributed Deep Learning in a Supercomputing Environmentcited by 1
  13. 2023Accelerating Wireless Federated Learning via Nesterov's Momentum and Distributed Principle Component Analysiscited by 1
  14. 2023GQFedWAvg: Optimization-Based Quantized Federated Learning in General Edge Computing Systemscited by 1
  15. 2024Federated Learning-based Collaborative Wideband Spectrum Sensing and Scheduling for UAVs in UTM Systemscited by 1
  16. 2021Unit-Modulus Wireless Federated Learning Via Penalty Alternating Minimizationcited by 0
  17. 2022Random Orthogonalization for Federated Learning in Massive MIMO Systemscited by 0
  18. 2024Efficient Wireless Federated Learning via Low-Rank Gradient Factorizationcited by 0
  19. 2024Enhancing Federated Learning with Adaptive Differential Privacy and Priority-Based Aggregationcited by 0
  20. 2025Mobility-Aware Asynchronous Federated Learning with Dynamic Sparsificationcited by 0