mlsanityData to 7 Oct 2026

Parameter-free Mirror Descent

arXiv 2203.00444 ↗First posted 26 Feb 2022cs.LG math.OC stat.MLcited by 45

Not on the map. Why?

Abstract (arXiv)

We develop a modified online mirror descent framework that is suitable for building adaptive and parameter-free algorithms in unbounded domains. We leverage this technique to develop the first unconstrained online linear optimization algorithm achieving an optimal dynamic regret bound, and we further demonstrate that natural strategies based on Follow-the-Regularized-Leader are unable to achieve similar results. We also apply our mirror descent framework to build new parameter-free implicit updates, as well as a simplified and improved unconstrained scale-free algorithm.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 19 of its 32 references that are in the map. First posted on arXiv 26 Feb 2022.

Builds on

19 of 32 references.

References · 19

  1. 2019Introduction to Online Convex Optimizationcited by 793
  2. 2019Online Learning: A Modern Introduction Using Convex Optimizationcited by 407
  3. 2018Adaptive Online Learning in Dynamic Environmentscited by 130
  4. 2018Black-Box Reductions for Parameter-free Online Learning in Banach Spacescited by 124
  5. 2015Strongly Adaptive Online Learningcited by 102
  6. 2015Online Optimization : Competing with Dynamic Comparatorscited by 99
  7. 2020Dynamic Regret of Convex and Smooth Functionscited by 90
  8. 2016Scale-Free Online Learningcited by 83
  9. 2018Stochastic (Approximate) Proximal Point Methods: Convergence, Optimality, and Adaptivitycited by 61
  10. 2020Lipschitz and Comparator-Norm Adaptivity in Online Learningcited by 57
  11. 2021Impossible Tuning Made Possible: A New Expert Algorithm and Its Applicationscited by 52
  12. 2016Improved Strongly Adaptive Online Learning using Coin Bettingcited by 48
7 more
  1. 2017Online Learning Without Prior Informationcited by 36
  2. 2021Parameter-free Stochastic Optimization of Variationally Coherent Functionscited by 27
  3. 2015Adaptive Online Learningcited by 23
  4. 2019Combining Online Learning Guaranteescited by 22
  5. 2020Online mirror descent and dual averaging: keeping pace in the dynamic casecited by 21
  6. 2020Temporal Variability in Implicit Online Learningcited by 20
  7. 2019Matrix-Free Preconditioning in Online Learningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 12; 2024: 9; 2025: 9; 2026: 11.

45 papers.

Sort:

  1. 2023DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedulecited by 85
  2. 2023Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversioncited by 42
  3. 2022Adapting to Online Label Shift with Provable Guaranteescited by 26
  4. 2023Unconstrained Online Learning with Unbounded Lossescited by 25
  5. 2024Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguisecited by 23
  6. 2023Optimistic Online Mirror Descent for Bridging Stochastic and Adversarial Online Convex Optimizationcited by 22
  7. 2023Mechanic: A Learning Rate Tunercited by 20
  8. 2024Random Scaling and Momentum for Non-smooth Non-convex Optimizationcited by 18
  9. 2024Fast TRAC: A Parameter-Free Optimizer for Lifelong Reinforcement Learningcited by 16
  10. 2023Efficient Methods for Non-stationary Online Learningcited by 15
  11. 2024Discounted Adaptive Online Learning: Towards Better Regularizationcited by 12
  12. 2023Unconstrained Dynamic Regret via Sparse Codingcited by 11
33 more
  1. 2023Online Label Shift: Optimal Dynamic Regret meets Practical Algorithmscited by 11
  2. 2024Gradient-Variation Online Learning under Generalized Smoothnesscited by 11
  3. 2024Online Linear Regression in Dynamic Environments via Discountingcited by 10
  4. 2024Fully Unconstrained Online Learningcited by 10
  5. 2022Optimal Comparator Adaptive Online Learning with Switching Costcited by 9
  6. 2024An Equivalence Between Static and Dynamic Regret Minimizationcited by 9
  7. 2023Improving Adaptive Online Learning Using Refined Discretizationcited by 8
  8. 2022Second Order Path Variationals in Non-Stationary Online Learningcited by 7
  9. 2022Optimal Dynamic Regret in LQR Controlcited by 6
  10. 2024Universal Online Convex Optimization with 1 Projection per Roundcited by 6
  11. 2025Heavy-Tailed Linear Bandits: Huber Regression with One-Pass Updatecited by 5
  12. 2025On the Dynamic Regret of Following the Regularized Leader: Optimism with History Pruningcited by 5
  13. 2023μ2-SGD: Stable Stochastic Optimization via a Double Momentum Mechanismcited by 4
  14. 2025Dynamic Regret Reduces to Kernelized Static Regretcited by 4
  15. 2025Stacey: Promoting Stochastic Steepest Descent via Accelerated ℓp-Smooth Nonconvex Optimizationcited by 3
  16. 2025Universal Dynamic Regret and Constraint Violation Bounds for Constrained Online Convex Optimizationcited by 3
  17. 2025Parameter-free Algorithms for the Stochastically Extended Adversarial Modelcited by 3
  18. 2023Parameter-free projected gradient descentcited by 2
  19. 2026Partially Lazy Gradient Descent for Smoothed Online Learningcited by 2
  20. 2026Parameter-free Dynamic Regret: Time-varying Movement Costs, Delayed Feedback, and Memorycited by 2
  21. 2026Dynamic Regret via Discounted-to-Dynamic Reduction with Applications to Curved Losses and Adam Optimizercited by 2
  22. 2026Parameter-Free Dynamic Regret for Unconstrained Linear Banditscited by 2
  23. 2025The Sample Complexity of Parameter-Free Stochastic Convex Optimizationcited by 1
  24. 2025Stochastic Shortest Path with Sparse Adversarial Costscited by 1
  25. 2026Gradient-Variation Regret Bounds for Unconstrained Online Learningcited by 1
  26. 2026Online Learning with Gradient-Variation Interval Regretcited by 1
  27. 2023Oblivious Stochastic Composite Optimizationcited by 0
  28. 2025A regret minimization approach to fixed-point iterationscited by 0
  29. 2026High-Probability Minimax Adaptive Estimation in Besov Spaces via Online-to-Batchcited by 0
  30. 2026A Perturbation Approach to Unconstrained Linear Banditscited by 0
  31. 2026Clipping the Price of Adaptivity at the Tailcited by 0
  32. 2026Parameter-Free Dynamic Regret under Heavy-Tailed Noisecited by 0
  33. 2026Dynamic Regret in Online Convex Optimization with Indicator Switching Costscited by 0