# Convergence Rates of Finite Difference Stochastic Approximation Algorithms

arXiv: https://arxiv.org/abs/1506.09211 · HTML: https://mlsanity.com/paper/1506.09211/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2015.

## Built on by (most cited first, 2 of 2)

- [On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization](https://mlsanity.com/paper/2510.19953/) (2025)
- [BOND: License to Train with Black-Box Functions](https://mlsanity.com/paper/2510.25074/) (2025)

## Abstract (arXiv)

Recently there has been renewed interests in derivative free approaches to stochastic optimization. In this paper, we examine the rates of convergence for the Kiefer-Wolfowitz algorithm and the mirror descent algorithm, under various updating schemes using finite differences as gradient approximations. It is shown that the convergence of these algorithms can be accelerated by controlling the implementation of the finite differences. Particularly, it is shown that the rate can be increased to n-2/5 in general and to n-1/2 in Monte Carlo optimization for a broad class of problems, in the iteration number n.