# Asynchronous Gradient-Push

arXiv: https://arxiv.org/abs/1803.08950 · HTML: https://mlsanity.com/paper/1803.08950/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Mar 2018.

## Built on by (most cited first, 20 of 20)

- [FedAvg with Fine Tuning: Local Updates Lead to Representation Learning](https://mlsanity.com/paper/2205.13692/) (2022)
- [Stochastic Gradient Descent under Markovian Sampling Schemes](https://mlsanity.com/paper/2302.14428/) (2023)
- [ResIST: Layer-Wise Decomposition of ResNets for Distributed Training](https://mlsanity.com/paper/2107.00961/) (2021)
- [Asynchronous SGD on Graphs: a Unified Framework for Asynchronous Decentralized and Federated Optimization](https://mlsanity.com/paper/2311.00465/) (2023)
- [Decentralized Directed Collaboration for Personalized Federated Learning](https://mlsanity.com/paper/2405.17876/) (2024)
- [Boosting Asynchronous Decentralized Learning with Model Fragmentation](https://mlsanity.com/paper/2410.12918/) (2024)
- [Asymmetrically Decentralized Federated Learning](https://mlsanity.com/paper/2310.05093/) (2023)
- [DRACO: Decentralized Asynchronous Federated Learning over Row-Stochastic Wireless Networks](https://mlsanity.com/paper/2406.13533/) (2024)
- [Asynchronous speedup in decentralized optimization](https://mlsanity.com/paper/2106.03585/) (2021)
- [Straggler-Resilient Decentralized Learning via Adaptive Asynchronous Updates](https://mlsanity.com/paper/2306.06559/) (2023)
- [Convergence Analysis of Decentralized ASGD](https://mlsanity.com/paper/2309.03754/) (2023)
- [Asynchronous Distributed Optimization with Delay-free Parameters](https://mlsanity.com/paper/2312.06508/) (2023)
- [Asynchronous Diffusion Learning with Agent Subsampling and Local Updates](https://mlsanity.com/paper/2402.05529/) (2024)
- [Asyn2F: An Asynchronous Federated Learning Framework with Bidirectional Model Aggregation](https://mlsanity.com/paper/2403.01417/) (2024)
- [Resource-Constrained Decentralized Federated Learning via Personalized Event-Triggering](https://mlsanity.com/paper/2211.12640/) (2022)
- [Adaptive Consensus: A network pruning approach for decentralized optimization](https://mlsanity.com/paper/2309.02626/) (2023)
- [LAYUP: Asynchronous decentralized gradient descent with LAYer-wise UPdates](https://mlsanity.com/paper/2410.05985/) (2024)
- [A Tale of Two Learning Algorithms: Multiple Stream Random Walk and Asynchronous Gossip](https://mlsanity.com/paper/2504.09792/) (2025)
- [ADP-VRSGP: Decentralized Learning with Adaptive Differential Privacy via Variance-Reduced Stochastic Gradient Push](https://mlsanity.com/paper/2510.20157/) (2025)
- [Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs](https://mlsanity.com/paper/2602.20567/) (2026)

## Abstract (arXiv)

We consider a multi-agent framework for distributed optimization where each agent has access to a local smooth strongly convex function, and the collective goal is to achieve consensus on the parameters that minimize the sum of the agents' local functions. We propose an algorithm wherein each agent operates asynchronously and independently of the other agents. When the local functions are strongly-convex with Lipschitz-continuous gradients, we show that the iterates at each agent converge to a neighborhood of the global minimum, where the neighborhood size depends on the degree of asynchrony in the multi-agent network. When the agents work at the same rate, convergence to the global minimizer is achieved. Numerical experiments demonstrate that Asynchronous Gradient-Push can minimize the global objective faster than state-of-the-art synchronous first-order methods, is more robust to failing or stalling agents, and scales better with the network size.