mlsanityData to 7 Oct 2026

LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learning

arXiv 1805.09965 ↗First posted 25 May 2018stat.ML cs.DC cs.LGcited by 95

Not on the map. Why?

Abstract (arXiv)

This paper presents a new class of gradient methods for distributed machine learning that adaptively skip the gradient calculations to learn with reduced communication and computation. Simple rules are designed to detect slowly-varying gradients and, therefore, trigger the reuse of outdated gradients. The resultant gradient-based algorithms are termed Lazily Aggregated Gradient --- justifying our acronym LAG used henceforth. Theoretically, the merits of this contribution are: i) the convergence rate is the same as batch gradient descent in strongly-convex, convex, and nonconvex smooth cases; and, ii) if the distributed datasets are heterogeneous (quantified by certain measurable constants), the communication rounds needed to achieve a targeted accuracy are reduced thanks to the adaptive reuse of lagged gradients. Numerical experiments on both synthetic and real data corroborate a significant communication reduction compared to alternatives.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 28; 2022: 29; 2023: 16; 2024: 16; 2025: 5; 2026: 1.

95 papers.

Sort:

  1. 2021Distributed Learning in Wireless Networks: Recent Progress and Future Challengescited by 125
  2. 2021Decentralized Federated Averagingcited by 114
  3. 2021CAFE: Catastrophic Data Leakage in Vertical Federated Learningcited by 76
  4. 2021Exponential Graph is Provably Efficient for Decentralized Deep Trainingcited by 55
  5. 2022Federated Learning via Inexact ADMMcited by 44
  6. 2021Anarchic Federated Learningcited by 42
  7. 2021DecentLaM: Decentralized Momentum SGD for Large-batch Deep Trainingcited by 40
  8. 2021Dynamic Scheduling for Over-the-Air Federated Edge Learning with Energy Constraintscited by 34
  9. 2022Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compressioncited by 33
  10. 2022Federated Learning Under Intermittent Client Availability and Time-Varying Communication Constraintscited by 30
  11. 20223PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregationcited by 27
  12. 2021Accelerating Gossip SGD with Periodic Global Averagingcited by 25
38 more
  1. 2022FedNew: A Communication-Efficient and Privacy-Preserving Newton-Type Method for Federated Learningcited by 23
  2. 2022AsyncFedED: Asynchronous Federated Learning with Euclidean Distance based Adaptive Weight Aggregationcited by 19
  3. 2021Asynchronous Iterations in Optimization: New Sequence Results and Sharper Algorithmic Guaranteescited by 17
  4. 2021BlueFog: Make Decentralized Algorithms Practical for Optimization and Deep Learningcited by 16
  5. 2021Adaptive Transmission Scheduling in Wireless Networks for Asynchronous Federated Learningcited by 15
  6. 2021Metadata Normalizationcited by 14
  7. 2021Accelerated Gradient Descent Learning over Multiple Access Fading Channelscited by 14
  8. 2021Communication-Efficient ADMM-based Federated Learningcited by 14
  9. 2022Adaptive Compression for Communication-Efficient Distributed Trainingcited by 14
  10. 2023Unbiased Compression Saves Communication in Distributed Optimization: When and How Much?cited by 14
  11. 2021Joint Optimization of Communications and Federated Learning Over the Aircited by 13
  12. 2021Federated Learning over Wireless IoT Networks with Optimized Communication and Resourcescited by 13
  13. 2021BROADCAST: Reducing Both Stochastic and Compression Noise to Robustify Communication-Efficient Federated Learningcited by 12
  14. 2022SHED: A Newton-type algorithm for federated learning based on incremental Hessian eigenvector sharingcited by 11
  15. 2022Distributed Newton-Type Methods with Communication Compression and Bernoulli Aggregationcited by 11
  16. 2023Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compressioncited by 11
  17. 2021Bandwidth Allocation for Multiple Federated Learning Services in Wireless Edge Networkscited by 10
  18. 20211-Bit Compressive Sensing for Efficient Federated Learning Over the Aircited by 10
  19. 2021Joint Client Scheduling and Resource Allocation under Channel Uncertainty in Federated Learningcited by 10
  20. 2023Over-the-Air Federated Learning and Optimizationcited by 10
  21. 2022Distributed Learning With Sparsified Gradient Differencescited by 9
  22. 2022A Snapshot of the Frontiers of Client Selection in Federated Learningcited by 9
  23. 2024Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Surveycited by 9
  24. 2022FedGiA: An Efficient Hybrid Algorithm for Federated Learningcited by 8
  25. 2022Fast Heterogeneous Federated Learning with Hybrid Client Selectioncited by 8
  26. 2021Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimationcited by 7
  27. 2023AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection Strategycited by 7
  28. 2021BEV-SGD: Best Effort Voting SGD for Analog Aggregation Based Federated Learning against Byzantine Attackerscited by 6
  29. 2022Accelerating Asynchronous Federated Learning Convergence via Opportunistic Mobile Relayingcited by 6
  30. 2022Personalized Federated Learning with Communication Compressioncited by 6
  31. 2022Communication-Efficient Actor-Critic Methods for Homogeneous Markov Gamescited by 5
  32. 2023Private Federated Learning with Autotuned Compressioncited by 5
  33. 2023Linear Speedup of Incremental Aggregated Gradient Methods on Streaming Datacited by 5
  34. 2024Version age-based client scheduling policy for federated learningcited by 5
  35. 2022Accelerating Federated Edge Learning via Topology Optimizationcited by 4
  36. 2022FedCau: A Proactive Stop Policy for Communication and Computation Efficient Federated Learningcited by 4
  37. 2022Clustered Federated Learning based on Nonconvex Pairwise Fusioncited by 4
  38. 2024Rendering Wireless Environments Useful for Gradient Estimators: A Zero-Order Stochastic Federated Learning Methodcited by 4

50 most cited of 95 All 95 (JSON)