mlsanityData to 14 Oct 2026

Effectiveness of Constant Stepsize in Markovian LSA and Statistical Inference

arXiv 2312.10894 ↗First posted 18 Dec 2023stat.ML cs.LG stat.MEcited by 5

Not on the map. Why?

Abstract (arXiv)

In this paper, we study the effectiveness of using a constant stepsize in statistical inference via linear stochastic approximation (LSA) algorithms with Markovian data. After establishing a Central Limit Theorem (CLT), we outline an inference procedure that uses averaged LSA iterates to construct confidence intervals (CIs). Our procedure leverages the fast mixing property of constant-stepsize LSA for better covariance estimation and employs Richardson-Romberg (RR) extrapolation to reduce the bias induced by constant stepsize and Markovian data. We develop theoretical results for guiding stepsize selection in RR extrapolation, and identify several important settings where the bias provably vanishes even without extrapolation. We conduct extensive numerical experiments and compare against classical inference approaches. Our results show that using a constant stepsize enjoys easy hyperparameter tuning, fast convergence, and consistently better CI coverage, especially when data is limited.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 16 of its 32 references that are in the map. First posted on arXiv 18 Dec 2023.

Builds on

16 of 32 references.

References · 16

  1. 2018A Finite Time Analysis of Temporal Difference Learning With Linear Function Approximationcited by 229
  2. 2019Finite-Time Error Bounds For Linear Stochastic Approximation and TD Learningcited by 166
  3. 2017Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chainscited by 88
  4. 2016Statistical Inference for Model Parameters in Stochastic Gradient Descentcited by 71
  5. 2020On Linear Stochastic Approximation: Fine-grained Polyak-Ruppert and Non-Asymptotic Concentrationcited by 68
  6. 2020Least Squares Regression with Markovian Data: Fundamental Limits and Algorithmscited by 56
  7. 2020An Analysis of Constant Step Size SGD in the Non-convex Regime: Asymptotic Normality and Biascited by 48
  8. 2022Finite-time High-probability Bounds for Polyak-Ruppert Averaged Iterates of Linear Stochastic Approximationcited by 41
  9. 2020Online Covariance Matrix Estimation in Stochastic Gradient Descentcited by 37
  10. 2021Optimal and instance-dependent guarantees for Markovian linear stochastic approximationcited by 33
  11. 2021The ODE Method for Asymptotic Statistics in Stochastic Approximation and Reinforcement Learningcited by 32
  12. 2022Bias and Extrapolation in Markovian Linear Stochastic Approximation with Constant Stepsizescited by 30
4 more
  1. 2021Fast and Robust Online Inference with Stochastic Gradient Descent via Random Scalingcited by 28
  2. 2017Statistical inference using SGDcited by 21
  3. 2023Online Statistical Inference for Nonlinear Stochastic Approximation with Markovian Datacited by 20
  4. 2023Statistical Inference with Stochastic Gradient Methods under φ-mixing Datacited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 0; 2026: 1.

5 papers.

  1. 2024Nonasymptotic Analysis of Stochastic Gradient Descent with the Richardson-Romberg Extrapolationcited by 6
  2. 2024Two-Timescale Linear Stochastic Approximation: Constant Stepsizes Go a Long Waycited by 5
  3. 2024Prelimit Coupling and Steady-State Convergence of Constant-stepsize Nonsmooth Contractive SAcited by 3
  4. 2026Self-Normalized Inference for Constant-Stepsize Temporal-Difference Learning under Markovian Samplingcited by 1
  5. 2024Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsizecited by 0