# Variance-Reduced Decentralized Stochastic Optimization with Gradient Tracking -- Part II: GT-SVRG

arXiv: https://arxiv.org/abs/1910.04057 · HTML: https://mlsanity.com/paper/1910.04057/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2019.

## Built on by (most cited first, 2 of 2)

- [GT-STORM: Taming Sample, Communication, and Memory Complexities in Decentralized Non-Convex Learning](https://mlsanity.com/paper/2105.01231/) (2021)
- [Distributed Random Reshuffling over Networks](https://mlsanity.com/paper/2112.15287/) (2021)

## Abstract (arXiv)

Decentralized stochastic optimization has recently benefited from gradient tracking methods citeDSGTPu,DSGTXin providing efficient solutions for large-scale empirical risk minimization problems. In Part I citeGTSAGA of this work, we develop textbftextttGT-SAGA that is based on a decentralized implementation of SAGA citeSAGA using gradient tracking and discuss regimes of practical interest where textbftextttGT-SAGA outperforms existing decentralized approaches in terms of the total number of local gradient computations. In this paper, we describe textbftextttGT-SVRG that develops a decentralized gradient tracking based implementation of SVRG citeSVRG, another well-known variance-reduction technique. We show that the convergence rate of textbftextttGT-SVRG matches that of textbftextttGT-SAGA for smooth and strongly-convex functions and highlight different trade-offs between the two algorithms in various settings.