mlsanityData to 7 Oct 2026

Fully Unconstrained Online Learning

arXiv 2405.20540 ↗First posted 30 May 2024cs.LG math.OC stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

We provide an online learning algorithm that obtains regret G|w_star|√Tlog(|w_star|G√T) + |w_star|2 + G2 on G-Lipschitz convex losses for any comparison point w_star without knowing either G or |w_star|. Importantly, this matches the optimal bound G|w_star|√T available with such knowledge (up to logarithmic factors), unless either |w_star| or G is so large that even G|w_star|√T is roughly linear in T. Thus, it matches the optimal bound in all cases in which one can achieve sublinear regret, which arguably most "interesting" scenarios.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 30 of its 45 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

30 of 45 references.

References · 30

  1. 2019Introduction to Online Convex Optimizationcited by 793
  2. 2019Online Learning: A Modern Introduction Using Convex Optimizationcited by 407
  3. 2016Coin Betting and Parameter-Free Online Learningcited by 131
  4. 2023Prodigy: An Expeditiously Adaptive Parameter-Free Learnercited by 129
  5. 2018Black-Box Reductions for Parameter-free Online Learning in Banach Spacescited by 124
  6. 2023Learning-Rate-Free Learning by D-Adaptationcited by 113
  7. 2023DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedulecited by 85
  8. 2020Lipschitz and Comparator-Norm Adaptivity in Online Learningcited by 57
  9. 2019UniXGrad: A Universal, Adaptive Algorithm with Optimal Guarantees for Constrained Optimizationcited by 56
  10. 2021Impossible Tuning Made Possible: A New Expert Algorithm and Its Applicationscited by 52
  11. 2022Making SGD Parameter-Freecited by 50
  12. 2022Parameter-free Mirror Descentcited by 45
18 more
  1. 2021Tight Concentrations and Confidence Sequences from the Regret of Universal Portfoliocited by 42
  2. 2023Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversioncited by 42
  3. 2023Optimal Linear Decay Learning Rate Schedules and Further Refinementscited by 41
  4. 2017Online Learning Without Prior Informationcited by 36
  5. 2019Lipschitz Adaptivity with Multiple Learning Rates in Online Learningcited by 36
  6. 2019Parameter-Free Online Convex Optimization with Sub-Exponential Noisecited by 28
  7. 2022PDE-Based Optimal Strategy for Unconstrained Online Learningcited by 27
  8. 2022Parameter-free Regret in High Probability with Heavy Tailscited by 26
  9. 2023Unconstrained Online Learning with Unbounded Lossescited by 25
  10. 2023Mechanic: A Learning Rate Tunercited by 20
  11. 2022Scale-free Unconstrained Online Learning for Curved Lossescited by 13
  12. 2024The Price of Adaptivity in Stochastic Convex Optimizationcited by 11
  13. 2020Risk-Monotonicity in Statistical Learningcited by 9
  14. 2022Damped Online Newton Step for Portfolio Selectioncited by 9
  15. 2022Quasi-Newton Steps for Efficient Online Exp-Concave Optimizationcited by 9
  16. 2023Improving Adaptive Online Learning Using Refined Discretizationcited by 8
  17. 2019Matrix-Free Preconditioning in Online Learningcited by 5
  18. 2023Projection-Free Online Convex Optimization via Efficient Newton Iterationscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 5.

10 papers.

  1. 2025Dynamic Regret Reduces to Kernelized Static Regretcited by 4
  2. 2026Simple Projection-Free Algorithm for Contextual Recommendation with Logarithmic Regret and Robustnesscited by 4
  3. 2025Sparsity-Based Interpolation of External, Internal and Swap Regretcited by 3
  4. 2025Parameter-free Algorithms for the Stochastically Extended Adversarial Modelcited by 3
  5. 2026Parameter-free Dynamic Regret: Time-varying Movement Costs, Delayed Feedback, and Memorycited by 2
  6. 2025Instance-Optimal Matrix Multiplicative Weight Update and Its Quantum Applicationscited by 1
  7. 2026Gradient-Variation Regret Bounds for Unconstrained Online Learningcited by 1
  8. 2025A regret minimization approach to fixed-point iterationscited by 0
  9. 2026Small Gradient Norm Regret for Online Convex Optimizationcited by 0
  10. 2026Online Prediction of Stochastic Sequences with High Probability Regret Boundscited by 0