# A Zeroth-Order Variance-Reduced Method for Decentralized Stochastic Non-convex Optimization

arXiv: https://arxiv.org/abs/2310.18883 · HTML: https://mlsanity.com/paper/2310.18883/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2023.

## Built on by (most cited first, 1 of 1)

- [Memory Savings at What Cost? A Study of Alternatives to Backpropagation](https://mlsanity.com/paper/2506.21833/) (2025)

## Abstract (arXiv)

In this paper, we consider a distributed stochastic non-convex optimization problem, which is about minimizing a sum of n local cost functions over a network with only zeroth-order information. A novel single-loop Decentralized Zeroth-Order Variance Reduction algorithm, called DZOVR, is proposed, which combines two-point gradient estimation, momentum-based variance reduction technique, and gradient tracking. Under mild assumptions, we show that the algorithm is able to achieve O(dn-1ε-3) sampling complexity at each node to reach an ε-accurate stationary point and also exhibits network-independent and linear speedup properties. To the best of our knowledge, this is the first stochastic decentralized zeroth-order algorithm that achieves this sampling complexity. Numerical experiments demonstrate that DZOVR outperforms the other state-of-the-art algorithms and has network-independent and linear speedup properties.