mlsanityData to 7 Oct 2026

Normalized Gradients for All

arXiv 2308.05621 ↗First posted 10 Aug 2023cs.LG math.OC stat.MLcited by 18

Not on the map. Why?

Abstract (arXiv)

In this short note, I show how to adapt to H"older smoothness using normalized gradients in a black-box way. Moreover, the bound will depend on a novel notion of local H"older smoothness. The main idea directly comes from Levy [2017].

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 10 Aug 2023.

Builds on

24 of 36 references.

References · 24

  1. 2019Online Learning: A Modern Introduction Using Convex Optimizationcited by 407
  2. 2018On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizescited by 155
  3. 2016Coin Betting and Parameter-Free Online Learningcited by 131
  4. 2018Black-Box Reductions for Parameter-free Online Learning in Banach Spacescited by 124
  5. 2023DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedulecited by 85
  6. 2019Adaptive Gradient Descent without Descentcited by 79
  7. 2020Lipschitz and Comparator-Norm Adaptivity in Online Learningcited by 57
  8. 2022Making SGD Parameter-Freecited by 50
  9. 2014Unconstrained Online Linear Learning in Hilbert Spaces: Minimax Algorithms and Normal Approximationscited by 46
  10. 2017Online to Offline Conversions, Universality and Adaptive Minibatch Sizescited by 39
  11. 2017Online Learning Without Prior Informationcited by 36
  12. 2023DoWG Unleashed: An Efficient Universal Parameter-Free Gradient Descent Methodcited by 35
12 more
  1. 2017Parameter-free online learning via model selectioncited by 34
  2. 2015Scale-Free Algorithms for Online Linear Optimizationcited by 30
  3. 2019Parameter-Free Online Convex Optimization with Sub-Exponential Noisecited by 28
  4. 2014Simultaneous Model Selection and Optimization through Parameter-free Stochastic Learningcited by 27
  5. 2021Parameter-free Stochastic Optimization of Variationally Coherent Functionscited by 27
  6. 2022PDE-Based Optimal Strategy for Unconstrained Online Learningcited by 27
  7. 2022Parameter-free Regret in High Probability with Heavy Tailscited by 26
  8. 2023Unconstrained Online Learning with Unbounded Lossescited by 25
  9. 2017Convergence Rates for Deterministic and Stochastic Subgradient Methods Without Lipschitz Continuitycited by 18
  10. 2022On Optimal Universal First-Order Methods for Minimizing Heterogeneous Sumscited by 9
  11. 2022Implicit Parameter-free Online Learning with Truncated Linear Modelscited by 8
  12. 2023Parameter-free projected gradient descentcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 9; 2026: 5.

18 papers.

Sort:

  1. 2025Training Deep Learning Models with Norm-Constrained LMOscited by 177
  2. 2025AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updatescited by 22
  3. 2023A simple uniformly optimal method without line search for convex optimizationcited by 18
  4. 2025Generalized Gradient Norm Clipping & Non-Euclidean (L0,L1)-Smoothnesscited by 17
  5. 2024Directional Smoothness and Gradient Methods: Convergence and Adaptivitycited by 15
  6. 2024Tuning-Free Stochastic Optimizationcited by 13
  7. 2024Discounted Adaptive Online Learning: Towards Better Regularizationcited by 12
  8. 2025SGD with Adaptive Preconditioning: Unified Analysis and Momentum Accelerationcited by 12
  9. 2025Gradient-Variation Online Adaptivity for Accelerated Optimization with H"older Smoothnesscited by 5
  10. 2025Nesterov Finds GRAAL: Optimal and Adaptive Gradient Method for Convex Optimizationcited by 4
  11. 2025Non-Euclidean Broximal Point Method: A Blueprint for Geometry-Aware Optimizationcited by 3
  12. 2025Glocal Smoothness: Line search and adaptive step sizes can help in theory too!cited by 2
6 more
  1. 2026Optimal Projection-Free Adaptive SGD for Matrix Optimizationcited by 1
  2. 2025Stochastic Adaptive Gradient Descent Without Descentcited by 0
  3. 2026Stochastic Auto-conditioned Fast Gradient Methods with Optimal Ratescited by 0
  4. 2026Stochastic Non-Smooth Convex Optimization with Unbounded Gradientscited by 0
  5. 2026AdaGrad does not adapt to H"older-smoothness for composite objectivescited by 0
  6. 2026Near-Optimal Acceleration for Smooth ℓp / ℓq Nondual Convex First-Order Oracle Optimizationcited by 0