# Scaling Limits of Constant-Stepsize SGD at Flat Minima

arXiv: https://arxiv.org/abs/2607.16384 · HTML: https://mlsanity.com/paper/2607.16384/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 47 references that are in the map.
- First posted on arXiv 17 Jul 2026.

## Builds on (most cited first, 3 of 3)

- [A Piecewise Lyapunov Analysis of Sub-quadratic SGD: Applications to Robust and Quantile Regression](https://mlsanity.com/paper/2504.08178/) (2025)
- [Constant-Stepsize Stochastic Approximation: Finite-Time Convergence, Gaussian Approximation, and Tail Bounds](https://mlsanity.com/paper/2602.13960/) (2026)
- [Revisiting the Constant Stepsize Stochastic Approximation with Decision-Dependent Markovian Noise](https://mlsanity.com/paper/2604.13378/) (2026)

## Abstract (arXiv)

For stochastic gradient descent (SGD) with a constant stepsize α, the invariant law of the iterates, centered at a minimizer, describes the behavior of the algorithm over long time horizons. In the strongly convex case, this invariant law has the familiar √α scaling and a Gaussian limit as αdownarrow 0. We show that this behavior changes fundamentally for convex objectives H with flat minima and (sub)quadratic tails. More specifically, we study SGD with Markovian noise generated by a contractive driving chain. For every sufficiently small constant stepsize α, we prove existence, uniqueness, and geometric convergence to an augmented invariant law in a Wasserstein distance induced by an α-dependent metric. When the minimizer x_star has local flatness exponent m≥2, meaning that nabla2 H(x)asymp lVert x-x_starrVertm-2 Id as x→ x_star, we obtain a contraction bound with factor 1-cαm-1, where c>0 is a constant. This recovers the factor 1-cα in the quadratic case m=2. We then analyze the small-stepsize scaling limit. We show that the invariant law concentrates on the scale α1/m and that the rescaled iterates converge weakly to the stationary distribution of the stochastic differential equation  dYt=-h0(Yt),dt+Sigma1/2,dBt ,  where h0 is the limiting drift at the minimizer and Sigma denotes the asymptotic covariance. This recovers the Gaussian limit when m=2 and gives generally non-Gaussian stationary limits in the flat case m>2. Finally, we give corresponding results for coordinate-separable objectives with unequal flatness exponents.