# Distributed stochastic optimization with gradient tracking over strongly-connected networks

arXiv: https://arxiv.org/abs/1903.07266 · HTML: https://mlsanity.com/paper/1903.07266/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2019.

## Built on by (most cited first, 11 of 11)

- [Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients](https://mlsanity.com/paper/2102.07053/) (2021)
- [An Improved Analysis of Gradient Tracking for Decentralized Machine Learning](https://mlsanity.com/paper/2202.03836/) (2022)
- [Decentralized Federated Learning with Gradient Tracking over Time-Varying Directed Networks](https://mlsanity.com/paper/2409.17189/) (2024)
- [Variance reduced stochastic optimization over directed graphs with row and column stochastic weights](https://mlsanity.com/paper/2202.03346/) (2022)
- [Zero-Order One-Point Estimate with Distributed Stochastic Gradient-Tracking Technique](https://mlsanity.com/paper/2210.05618/) (2022)
- [Robust Fully-Asynchronous Methods for Distributed Training over General Architecture](https://mlsanity.com/paper/2307.11617/) (2023)
- [Locally Differentially Private Gradient Tracking for Distributed Online Learning over Directed Graphs](https://mlsanity.com/paper/2310.16105/) (2023)
- [Competing Adaptive Networks](https://mlsanity.com/paper/2103.15664/) (2021)
- [Provably Accelerated Decentralized Gradient Method Over Unbalanced Directed Graphs](https://mlsanity.com/paper/2107.12065/) (2021)
- [The Impact of Adversarial Node Placement in Decentralized Federated Learning Networks](https://mlsanity.com/paper/2311.07946/) (2023)
- [Single Point-Based Distributed Zeroth-Order Optimization with a Non-Convex Stochastic Objective Function](https://mlsanity.com/paper/2410.05942/) (2024)

## Abstract (arXiv)

In this paper, we study distributed stochastic optimization to minimize a sum of smooth and strongly-convex local cost functions over a network of agents, communicating over a strongly-connected graph. Assuming that each agent has access to a stochastic first-order oracle (SFO), we propose a novel distributed method, called S-AB, where each agent uses an auxiliary variable to asymptotically track the gradient of the global cost in expectation. The S-AB algorithm employs row- and column-stochastic weights simultaneously to ensure both consensus and optimality. Since doubly-stochastic weights are not used, S-AB is applicable to arbitrary strongly-connected graphs. We show that under a sufficiently small constant step-size, S-AB converges linearly (in expected mean-square sense) to a neighborhood of the global minimizer. We present numerical simulations based on real-world data sets to illustrate the theoretical results.