# Simultaneous perturbation stochastic approximation: towards one-measurement per iteration

arXiv: https://arxiv.org/abs/2203.03075 · HTML: https://mlsanity.com/paper/2203.03075/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2022.

## Built on by (most cited first, 4 of 4)

- [Convergence Rates for Stochastic Approximation: Biased Noise with Unbounded Variance, and Applications](https://mlsanity.com/paper/2312.02828/) (2023)
- [MINI-LLM: Memory-Efficient Structured Pruning for Large Language Models](https://mlsanity.com/paper/2407.11681/) (2024)
- [Convergence of the Stochastic Heavy Ball Method With Approximate Gradients and/or Block Updating](https://mlsanity.com/paper/2303.16241/) (2023)
- [Revisiting Stochastic Approximation and Stochastic Gradient Descent](https://mlsanity.com/paper/2505.11343/) (2025)

## Abstract (arXiv)

When measuring the value of a function to be minimized is not only expensive but also with noise, the popular simultaneous perturbation stochastic approximation (SPSA) algorithm requires only two function values in each iteration. In this paper, we propose a method requiring only one function measurement value per iteration in the average sense. We prove the strong convergence and asymptotic normality of the new algorithm. Experimental results show the effectiveness and potential of our algorithm.