A stochastic subgradient method with optimal failure exponent

Fix a target accuracy varepsilon, a gradient-noise level s, and a horizon N. We wish to design algorithms which minimize the probability of observing a suboptimality gap which exceeds the target accuracy, i.e., EN = -log supf,P mathbbPP(f(xA) - f_star ≥ varepsilon), with the noise law P known only to be sub-Gaussian. We single out a uniformly averaged schedule which is harmonic (of the form hk = R2/(varepsilon (N+m-k))) and prove, via an optimized exponential supermartingale argument, that it attains the optimal exponent EN^star = varepsilon2 N (1+o(1))/(2R2 s2). Optimality is certified by a matching impossibility result: under Gaussian noise, a gradient-masking change of measure caps the exponent of every algorithm at the same leading order. In a small-noise limit, our setting degenerates into the adversarial-error model of G"osgens and van Parys (2025) for subgradient methods.