mlsanityData to 14 Oct 2026

Statistical inference using SGD

arXiv 1705.07477 ↗First posted 21 May 2017cs.LG cs.AI math.OCcited by 21

Not on the map. Why?

Abstract (arXiv)

We present a novel method for frequentist statistical inference in M-estimation problems, based on stochastic gradient descent (SGD) with a fixed step size: we demonstrate that the average of such SGD sequences can be used for statistical inference, after proper scaling. An intuitive analysis using the Ornstein-Uhlenbeck process suggests that such averages are asymptotically normal. From a practical perspective, our SGD-based inference procedure is a first order method, and is well-suited for large scale problems. To show its merits, we apply it to both synthetic and real datasets, and demonstrate that its accuracy is comparable to classical statistical methods, while requiring potentially far less computation.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 3; 2023: 7; 2024: 4; 2025: 2; 2026: 1.

21 papers.

Sort:

By area:

  1. 2021Online Bootstrap Inference For Policy Evaluation in Reinforcement Learningcited by 26
  2. 2023Online Statistical Inference for Nonlinear Stochastic Approximation with Markovian Datacited by 20
  3. 2021Bootstrapping the error of Oja's algorithmcited by 11
  4. 2022Statistical Inference of Constrained Stochastic Optimization via Sketched Sequential Quadratic Programmingcited by 10
  5. 2024High Confidence Level Inference is Almost Free using Parallel Stochastic Optimizationcited by 10
  6. 2023Weighted Averaged Stochastic Gradient Descent: Asymptotic Normality and Optimalitycited by 8
  7. 2023Online covariance estimation for stochastic gradient descent under Markovian samplingcited by 7
  8. 2023Effectiveness of Constant Stepsize in Markovian LSA and Statistical Inferencecited by 5
  9. 2022Tuning Stochastic Gradient Algorithms for Statistical Inference via Large-Sample Asymptoticscited by 4
  10. 2025Beyond Sin-Squared Error: Linear-Time Entrywise Uncertainty Quantification for Streaming PCAcited by 3
  11. 2021Online Statistical Inference for Parameters Estimation with Linear-Equality Constraintscited by 2
  12. 2021Statistical Estimation and Inference via Local SGD in Federated Learningcited by 2
9 more
  1. 2022Statistical inference with implicit SGD: proximal Robbins-Monro vs. Polyak-Ruppertcited by 2
  2. 2024Asymptotic Analysis of Sample-averaged Q-learningcited by 2
  3. 2023Statistical Inference for Linear Functionals of Online SGD in High-dimensional Linear Regressioncited by 1
  4. 2024Uncertainty quantification by block bootstrap for differentially private stochastic gradient descentcited by 1
  5. 2025Statistical Inference for Linear Functionals of Online Least-squares SGD when t gtrsim d1+δcited by 1
  6. 2023Doubly Stochastic Models: Learning with Unbiased Label Noises and Inference Stabilitycited by 0
  7. 2023Cheap Bootstrap for Fast Uncertainty Quantification of Stochastic Gradient Descentcited by 0
  8. 2024Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsizecited by 0
  9. 2026Refining Covariance Matrix Estimation in Stochastic Gradient Descent Through Bias Reductioncited by 0