Parameter-Free Dynamic Regret under Heavy-Tailed Noise
Not on the map. Why?
Abstract (arXiv)
We study online convex optimization with one unbiased stochastic subgradient per round and noise having a finite p-th central moment, where p∈(1,2] is unknown. For a bounded convex domain of diameter D, subgradients bounded by G, noise scale σ, and comparator path length PT, let LambdaT=1+PT/D. A single algorithm, using none of G,σ,p,PT, attains expected dynamic regret Opleft(minGD√TLambdaT+σ DT1/pLambdaT(p-1)/p,,GDTright) against every fixed comparator sequence. Restarted AdaGrad experts produce the noise-path exponent (p-1)/p, and a prior favoring longer restart intervals removes horizon-dependent logarithmic overhead. We give an explicit bound uniform in p; its logarithm-free form has noise coefficient O(1+log(p/(p-1))), while the static-regret constant is universal. The analysis requires only marginal noise moments and permits dependent errors. Complete pathwise proofs retain both the expert-loss range and the gradient energies preceding comparator movement. Matching lower bounds hold on every bounded convex domain of positive diameter, under the same gradient-only information model. Together with a path-budget-tuned upper bound, they characterize the minimax rate with universal constants, including its linear-regret saturation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 23 of its 35 references that are in the map. First posted on arXiv 29 Jul 2026.
Builds on
23 of 35 references.
References · 23
- 2019Introduction to Online Convex Optimizationcited by 803
- 2019Online Learning: A Modern Introduction Using Convex Optimizationcited by 417
- 2019A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networkscited by 214
- 2013Optimization, Learning, and Games with Predictable Sequencescited by 188
- 2013Non-stationary Stochastic Optimizationcited by 175
- 2018Adaptive Online Learning in Dynamic Environmentscited by 133
- 2015Strongly Adaptive Online Learningcited by 107
- 2020Dynamic Regret of Convex and Smooth Functionscited by 90
- 2016Scale-Free Online Learningcited by 85
- 2013Follow the Leader If You Can, Hedge If You Mustcited by 70
- 2021Adaptivity and Non-stationarity: Problem-dependent Dynamic Regret for Online Convex Optimizationcited by 66
- 2016Improved Strongly Adaptive Online Learning using Coin Bettingcited by 50
11 more
- 2016MetaGrad: Multiple Learning Rates in Online Learningcited by 47
- 2022Parameter-free Mirror Descentcited by 45
- 2024Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clippingcited by 42
- 2023Revisiting the Last-Iterate Convergence of Stochastic Gradient Methodscited by 38
- 2022Parameter-free Regret in High Probability with Heavy Tailscited by 28
- 2022New Projection-free Algorithms for Online Convex Optimization with Adaptive Regret Guaranteescited by 26
- 2022Mirror Descent Strikes Again: Optimal Stochastic Convex Optimization under Infinite Noise Variancecited by 24
- 2025Online Convex Optimization with Heavy Tails: Old Algorithms, New Regrets, and Applicationscited by 17
- 2022Optimal Comparator Adaptive Online Learning with Switching Costcited by 9
- 2024A Unified Framework for Analyzing Meta-algorithms in Online Convex Optimizationcited by 4
- 2024Online Distributed Optimization with Clipped Stochastic Gradients: High Probability Bound of Regretscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.