mlsanityData to 7 Oct 2026

Online Statistical Inference for Stochastic Optimization via Kiefer-Wolfowitz Methods

arXiv 2102.03389 ↗First posted 5 Feb 2021math.ST stat.ML stat.THcited by 14

Not on the map. Why?

Abstract (arXiv)

This paper investigates the problem of online statistical inference of model parameters in stochastic optimization problems via the Kiefer-Wolfowitz algorithm with random search directions. We first present the asymptotic distribution for the Polyak-Ruppert-averaging type Kiefer-Wolfowitz (AKW) estimators, whose asymptotic covariance matrices depend on the distribution of search directions and the function-value query complexity. The distributional result reflects the trade-off between statistical efficiency and function query complexity. We further analyze the choice of random search directions to minimize certain summary statistics of the asymptotic covariance matrix. Based on the asymptotic distribution, we conduct online statistical inference by providing two construction procedures of valid confidence intervals.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 11 of its 60 references that are in the map. First posted on arXiv 5 Feb 2021.

Builds on

11 of 60 references.

References · 11

  1. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  2. 2013Optimal rates for zero-order convex optimization: the power of two function evaluationscited by 190
  3. 2015An Optimal Algorithm for Bandit and Zero-Order Convex Optimization with Two-Point Feedbackcited by 105
  4. 2016Statistical Inference for Model Parameters in Stochastic Gradient Descentcited by 71
  5. 2017Stochastic Zeroth-order Optimization in High Dimensionscited by 62
  6. 2016Asymptotic Optimality in Stochastic Optimizationcited by 38
  7. 2020Online Covariance Matrix Estimation in Stochastic Gradient Descentcited by 37
  8. 2020Statistical Inference for Online Decision Making via Stochastic Gradient Descentcited by 17
  9. 2017Statistical Inference for the Population Landscape via Moment Adjusted Stochastic Gradientscited by 9
  10. 2019A generalization of regularized dual averaging and its dynamicscited by 6
  11. 2017On Scalable Inference with Stochastic Gradient Descentcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 3; 2024: 5; 2025: 3; 2026: 1.

14 papers.

Sort:

  1. 2021Fast and Robust Online Inference with Stochastic Gradient Descent via Random Scalingcited by 28
  2. 2023Online Statistical Inference for Nonlinear Stochastic Approximation with Markovian Datacited by 18
  3. 2024High Confidence Level Inference is Almost Free using Parallel Stochastic Optimizationcited by 9
  4. 2022Online Statistical Inference for Contextual Bandits via Stochastic Gradient Descentcited by 7
  5. 2023Online covariance estimation for stochastic gradient descent under Markovian samplingcited by 7
  6. 2023SGMM: Stochastic Approximation to Generalized Method of Momentscited by 3
  7. 2024Asymptotic Time-Uniform Inference for Parameters in Averaged Stochastic Approximationcited by 3
  8. 2025Statistical Inference for Online Algorithmscited by 2
  9. 2024Adaptive debiased SGD in high-dimensional GLMs with streaming datacited by 1
  10. 2024On the SAGA algorithm with decreasing stepcited by 1
  11. 2024Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsizecited by 0
  12. 2025A stochastic gradient descent algorithm with random search directionscited by 0
2 more
  1. 2025Online Statistical Inference of Constrained Stochastic Optimization via Random Scalingcited by 0
  2. 2026Statistical Inference on Gradient Flowscited by 0