# When Does Dynamic Preconditioning Preserve the Polyak-Ruppert CLT? A Stabilization Threshold

arXiv: https://arxiv.org/abs/2604.23498 · HTML: https://mlsanity.com/paper/2604.23498/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 46 references that are in the map.
- First posted on arXiv 26 Apr 2026.

## Builds on (most cited first, 9 of 9)

- [Gaussian Approximation and Multiplier Bootstrap for Stochastic Gradient Descent](https://mlsanity.com/paper/2502.06719/) (2025)
- [Acceleration of stochastic gradient descent with momentum by averaging: finite-sample rates and asymptotic normality](https://mlsanity.com/paper/2305.17665/) (2023)
- [Weighted Averaged Stochastic Gradient Descent: Asymptotic Normality and Optimality](https://mlsanity.com/paper/2307.06915/) (2023)
- [Improved Central Limit Theorem and Bootstrap Approximations for Linear Stochastic Approximation](https://mlsanity.com/paper/2510.12375/) (2025)
- [Online estimation of the inverse of the Hessian for stochastic optimization with application to universal stochastic Newton algorithms](https://mlsanity.com/paper/2401.10923/) (2024)
- [Finite-Sample Wasserstein Error Bounds and Concentration Inequalities for Nonlinear Stochastic Approximation](https://mlsanity.com/paper/2602.02445/) (2026)
- [High-dimensional limit theorems for SGD: Momentum and Adaptive Step-sizes](https://mlsanity.com/paper/2511.03952/) (2025)
- [Design Criteria for SGD Preconditioners: Local Conditioning, Noise Floors, and Basin Stability](https://mlsanity.com/paper/2511.19716/) (2025)
- [A Full Adagrad algorithm with O(Nd) operations](https://mlsanity.com/paper/2405.01908/) (2024)

## Built on by (most cited first, 1 of 1)

- [A Polyak-Ruppert Central Limit Theorem for SA-Adam with Momentum and Non-Convergent Adaptive Preconditioning](https://mlsanity.com/paper/2606.17364/) (2026)

## Abstract (arXiv)

Polyak-Ruppert averaging yields an asymptotically normal estimator with sandwich covariance H-1SH-1, the foundation of online inference. When the gradient step is preconditioned by a data-driven matrix Pt, we ask how fast Pt must stabilize for the central limit theorem (CLT) to remain valid. We resolve this via an exact preconditioner-isolating decomposition of the averaged error that confines Pt to a dynamic remainder Rn, leaving the martingale and Taylor terms preconditioner-free. Let Mt = (Pt H)-1 denote the effective inverse drift matrix, with |Mt - Mt-1|op lesssim t-β and step-size exponent α ∈ (1/2, 1). We identify a stabilization-rate threshold β > (α+1)/2 and prove that, within the class of polynomial rate hypotheses used in our upper bound, it cannot be weakened: the dynamic remainder √n,Rn vanishes in L2 whenever β > (α+1)/2, and we exhibit sequences satisfying those hypotheses for which it does not vanish when β ≤ (α+1)/2. A single stabilization argument certifies three SA variants - SA-AdaGrad, SA-RMSProp, and SA-ONS - with gain ρt = c/t, each delivering one-step L2(op) stabilization of order t-1, yielding the CLT √n(barxn - x^*) → N(0, H-1SH-1); under bounded inputs the pathwise rate β = 1 further preserves the n-1/6 Wasserstein rate at α^* = 2/3. Under standard regularity conditions, Wald-type online inference remains valid for dynamically preconditioned averaged SGD whose stabilization rate exceeds the threshold.