mlsanityData to 7 Oct 2026

Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradients

arXiv 1909.07588 ↗First posted 17 Sep 2019cs.LG cs.DC math.OCcited by 32

Not on the map. Why?

Abstract (arXiv)

The present paper develops a novel aggregated gradient approach for distributed machine learning that adaptively compresses the gradient communication. The key idea is to first quantize the computed gradients, and then skip less informative quantized gradient communications by reusing outdated gradients. Quantizing and skipping result in `lazy' worker-server communications, which justifies the term Lazily Aggregated Quantized gradient that is henceforth abbreviated as LAQ. Our LAQ can provably attain the same linear convergence rate as the gradient descent in the strongly convex case, while effecting major savings in the communication overhead both in transmitted bits as well as in communication rounds. Empirically, experiments with real data corroborate a significant communication reduction compared to existing gradient- and stochastic gradient-based algorithms.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 7; 2022: 11; 2023: 7; 2024: 6; 2025: 1.

32 papers.

Sort:

  1. 2021Distributed Learning in Wireless Networks: Recent Progress and Future Challengescited by 125
  2. 2021Decentralized Federated Averagingcited by 114
  3. 20211-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speedcited by 74
  4. 2021Adaptive Quantization of Model Updates for Communication-Efficient Federated Learningcited by 60
  5. 20223PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregationcited by 27
  6. 20211-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speedcited by 24
  7. 2022FedNew: A Communication-Efficient and Privacy-Preserving Newton-Type Method for Federated Learningcited by 23
  8. 2022GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexitycited by 17
  9. 2022Distributed Distributionally Robust Optimization with Non-Convex Objectivescited by 14
  10. 2022Adaptive Compression for Communication-Efficient Distributed Trainingcited by 14
  11. 2021BROADCAST: Reducing Both Stochastic and Compression Noise to Robustify Communication-Efficient Federated Learningcited by 12
  12. 2022Distributed Newton-Type Methods with Communication Compression and Bernoulli Aggregationcited by 11
20 more
  1. 2024DeMo: Decoupled Momentum Optimizationcited by 10
  2. 2022Distributed Linear Bandits under Communication Constraintscited by 8
  3. 2021Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimationcited by 7
  4. 2023A Communication-Efficient Adaptive Algorithm for Federated Learning under Cumulative Regretcited by 7
  5. 2022Personalized Federated Learning with Server-Side Informationcited by 6
  6. 2023Blockchain-Based Federated Learning: Incentivizing Data Sharing and Penalizing Dishonest Behaviorcited by 5
  7. 2024FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink and Downlink Adaptive Quantizationcited by 5
  8. 2022FedCau: A Proactive Stop Policy for Communication and Computation Efficient Federated Learningcited by 4
  9. 2023Federated Distributionally Robust Optimization with Non-Convex Objectives: Algorithm and Analysiscited by 4
  10. 2024Distributed Learning based on 1-Bit Gradient Coding in the Presence of Stragglerscited by 4
  11. 2022Federated Learning Using Three-Operator ADMMcited by 3
  12. 2024Decentralized Personalized Federated Learning based on a Conditional Sparse-to-Sparser Schemecited by 3
  13. 2022Lazy Queries Can Reduce Variance in Zeroth-order Optimizationcited by 2
  14. 2023FedDIP: Federated Learning with Extreme Dynamic Pruning and Incremental Regularizationcited by 2
  15. 2024Socialized Learning: A Survey of the Paradigm Shift for Edge Intelligence in Networked Systemscited by 2
  16. 2023SparDL: Distributed Deep Learning Training with Efficient Sparse Communicationcited by 0
  17. 2023Communication-Efficient Zeroth-Order Distributed Online Optimization: Algorithm, Theory, and Applicationscited by 0
  18. 2023Communication-Efficient Federated Learning via Regularized Sparse Random Networkscited by 0
  19. 2024Random Gradient Masking as a Defensive Measure to Deep Leakage in Federated Learningcited by 0
  20. 2025Regularized Top-k: A Bayesian Framework for Gradient Sparsificationcited by 0