# The Exact Time-Uniform Rate Frontier for Stochastic Gradient Descent on Smooth Convex Objectives

arXiv: https://arxiv.org/abs/2609.08537 · HTML: https://mlsanity.com/paper/2609.08537/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 19 references that are in the map.
- First posted on arXiv 8 Sep 2026.

## Builds on (most cited first, 10 of 10)

- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Time-uniform, nonparametric, nonasymptotic confidence sequences](https://mlsanity.com/paper/1810.08240/) (2018)
- [Tight Analyses for Non-Smooth Stochastic Gradient Descent](https://mlsanity.com/paper/1812.05217/) (2018)
- [Revisiting the Last-Iterate Convergence of Stochastic Gradient Methods](https://mlsanity.com/paper/2312.08531/) (2023)
- [Exact convergence rate of the last iterate in subgradient methods](https://mlsanity.com/paper/2307.11134/) (2023)
- [On the Last Iterate Convergence of Momentum Methods](https://mlsanity.com/paper/2102.07002/) (2021)
- [Stopping Rules for Stochastic Gradient Descent via Anytime-Valid Confidence Sequences](https://mlsanity.com/paper/2512.13123/) (2025)
- [The Anytime Convergence of Stochastic Gradient Descent with Momentum: From a Continuous-Time Perspective](https://mlsanity.com/paper/2310.19598/) (2023)
- [Breaking a Logarithmic Barrier in the Stopping Time Convergence Rate of Stochastic First-order Methods](https://mlsanity.com/paper/2506.23335/) (2025)
- [Revisiting Stochastic Gradient Descent for Strongly Convex Objectives: Tight Uniform-in-Time Bounds](https://mlsanity.com/paper/2508.20823/) (2025)

## Built on by (most cited first, 1 of 1)

- [Exact information accounting for SGD methods](https://mlsanity.com/paper/2610.00446/) (2026)

## Abstract (arXiv)

We study the time-uniform convergence of the raw iterate of standard stochastic gradient descent (SGD) for unconstrained smooth convex objectives. We prove that, under standard noise assumptions, the time-uniform convergence rate gets arbitrarily close to √log n / n but never reaches it. More specifically, we prove that for every positive, eventually nondecreasing sequence h satisfying h(n) = o(√n), a bound of order h(n)/√n, holding simultaneously for all n with probability at least 1-α and uniformly over the problem class, is achievable if and only if [ sumj = 1∞ frac1h(2j)2 < ∞. ] The constructive sufficiency result follows from a dyadic horizon-free schedule together with an additive conditional-restart inequality. The necessity counterpart applies to every deterministic nonnegative schedule and holds even for a one-dimensional analytic smooth convex objective with Gaussian noise.