mlsanityData to 14 Oct 2026

Trustworthy Efficient Communication for Distributed Learning using LQ-SGD Algorithm

arXiv 2506.17974 ↗First posted 22 Jun 2025cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

We propose LQ-SGD (Low-Rank Quantized Stochastic Gradient Descent), an efficient communication gradient compression algorithm designed for distributed training. LQ-SGD further develops on the basis of PowerSGD by incorporating the low-rank approximation and log-quantization techniques, which drastically reduce the communication overhead, while still ensuring the convergence speed of training and model accuracy. In addition, LQ-SGD and other compression-based methods show stronger resistance to gradient inversion than traditional SGD, providing a more robust and efficient optimization path for distributed learning systems.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 17 of its 21 references that are in the map. First posted on arXiv 22 Jun 2025.

Builds on

17 of 21 references.

References · 17

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  3. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  4. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 478
  5. 2018Sparsified SGD with Memorycited by 376
  6. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 310
  7. 2017Sparse Communication for Distributed Gradient Descentcited by 249
  8. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 238
  9. 2018Measuring the Effects of Data Parallelism on Neural Network Trainingcited by 193
  10. 2019A Distributed Synchronous SGD Algorithm with Global Top-k Sparsification for Low Bandwidth Networkscited by 37
  11. 2020Communication-Efficient Distributed Deep Learning: A Comprehensive Surveycited by 26
  12. 2025Models of Heavy-Tailed Mechanistic Universalitycited by 10
5 more
  1. 2024Demystifying the Communication Characteristics for Distributed Transformer Modelscited by 7
  2. 2021Communication efficient privacy-preserving distributed optimization using adaptive differential quantizationcited by 6
  3. 2023Adaptive Differentially Quantized Subspace Perturbation (ADQSP): A Unified Framework for Privacy-Preserving Distributed Average Consensuscited by 6
  4. 2024Trustworthiness of Stochastic Gradient Descent in Distributed Learningcited by 2
  5. 2025Lightweight Trustworthy Distributed Clusteringcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Evolution of Optimization Methods: Algorithms, Scenarios, and Evaluationscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2024Truncated Non-Uniform Quantization for Distributed SGD
  2. 2024Flattened one-bit stochastic gradient descent: compressed distributed optimization with controlled variance
  3. 2021DQ-SGD: Dynamic Quantization in SGD for Communication-Efficient Distributed Learning
  4. 2023Evaluation and Optimization of Gradient Compression for Distributed Deep Learning
  5. 2021NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization
  6. 2021FastSGD: A Fast Compressed SGD Framework for Distributed Machine Learning
  7. 2021Unbiased Single-scale and Multi-scale Quantizers for Distributed Optimization
  8. 2024Improved Quantization Strategies for Managing Heavy-tailed Gradients in Distributed Learning