# LASG: Lazily Aggregated Stochastic Gradients for Communication-Efficient Distributed Learning

arXiv: https://arxiv.org/abs/2002.11360 · HTML: https://mlsanity.com/paper/2002.11360/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2020.

## Built on by (most cited first, 5 of 5)

- [Anarchic Federated Learning](https://mlsanity.com/paper/2108.09875/) (2021)
- [Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimation](https://mlsanity.com/paper/2110.07751/) (2021)
- [A Novel Optimized Asynchronous Federated Learning Framework](https://mlsanity.com/paper/2111.09487/) (2021)
- [Adaptive Worker Grouping For Communication-Efficient and Straggler-Tolerant Distributed SGD](https://mlsanity.com/paper/2201.04301/) (2022)
- [Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning](https://mlsanity.com/paper/2610.15543/) (2026)

## Abstract (arXiv)

This paper targets solving distributed machine learning problems such as federated learning in a communication-efficient fashion. A class of new stochastic gradient descent (SGD) approaches have been developed, which can be viewed as the stochastic generalization to the recently developed lazily aggregated gradient (LAG) method --- justifying the name LASG. LAG adaptively predicts the contribution of each round of communication and chooses only the significant ones to perform. It saves communication while also maintains the rate of convergence. However, LAG only works with deterministic gradients, and applying it to stochastic gradients yields poor performance. The key components of LASG are a set of new rules tailored for stochastic gradients that can be implemented either to save download, upload, or both. The new algorithms adaptively choose between fresh and stale stochastic gradients and have convergence rates comparable to the original SGD. LASG achieves impressive empirical performance --- it typically saves total communication by an order of magnitude.