# A stochastic subgradient method with optimal failure exponent

arXiv: https://arxiv.org/abs/2609.37425 · HTML: https://mlsanity.com/paper/2609.37425/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 25 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Builds on (most cited first, 12 of 12)

- [Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming](https://mlsanity.com/paper/1309.5549/) (2013)
- [Tight Analyses for Non-Smooth Stochastic Gradient Descent](https://mlsanity.com/paper/1812.05217/) (2018)
- [High Probability Convergence of Stochastic Gradient Methods](https://mlsanity.com/paper/2302.14843/) (2023)
- [Asymptotic Optimality in Stochastic Optimization](https://mlsanity.com/paper/1612.05612/) (2016)
- [The Fragility of Optimized Bandit Algorithms](https://mlsanity.com/paper/2109.13595/) (2021)
- [Can SGD Handle Heavy-Tailed Noise?](https://mlsanity.com/paper/2508.04860/) (2025)
- [Negative Stepsizes Make Gradient-Descent-Ascent Converge](https://mlsanity.com/paper/2505.01423/) (2025)
- [Large deviations rates for stochastic gradient descent with strongly convex functions](https://mlsanity.com/paper/2211.00969/) (2022)
- [Large Deviation Upper Bounds and Improved MSE Rates of Nonlinear SGD: Heavy-tailed Noise and Power of Symmetry](https://mlsanity.com/paper/2410.15637/) (2024)
- [Risk level dependent Minimax Quantile lower bounds for Interactive Statistical Decision Making](https://mlsanity.com/paper/2510.05808/) (2025)
- [Subgradient Methods for Nonsmooth Convex Functions with Adversarial Errors](https://mlsanity.com/paper/2510.03072/) (2025)
- [Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization](https://mlsanity.com/paper/2602.05657/) (2026)

## Abstract (arXiv)

Fix a target accuracy varepsilon, a gradient-noise level s, and a horizon N. We wish to design algorithms which minimize the probability of observing a suboptimality gap which exceeds the target accuracy, i.e., EN = -log supf,P mathbbPP(f(xA) - f_star ≥ varepsilon), with the noise law P known only to be sub-Gaussian. We single out a uniformly averaged schedule which is harmonic (of the form hk = R2/(varepsilon (N+m-k))) and prove, via an optimized exponential supermartingale argument, that it attains the optimal exponent EN^star = varepsilon2 N (1+o(1))/(2R2 s2). Optimality is certified by a matching impossibility result: under Gaussian noise, a gradient-masking change of measure caps the exponent of every algorithm at the same leading order. In a small-noise limit, our setting degenerates into the adversarial-error model of G"osgens and van Parys (2025) for subgradient methods.