mlsanityData to 14 Oct 2026

Accelerated Parameter-Free Stochastic Optimization

arXiv 2404.00666 ↗First posted 31 Mar 2024cs.LG math.OCcited by 15

Not on the map. Why?

Abstract (arXiv)

We propose a method that achieves near-optimal rates for smooth stochastic convex optimization and requires essentially no prior knowledge of problem parameters. This improves on prior work which requires knowing at least the initial distance to optimality d0. Our method, U-DoG, combines UniXGrad (Kavis et al., 2019) and DoG (Ivgi et al., 2023) with novel iterate stabilization techniques. It requires only loose bounds on d0 and the noise magnitude, provides high probability guarantees under sub-Gaussian noise, and is also near-optimal in the non-smooth case. Our experiments show consistent, strong performance on convex problems and mixed results on neural network training.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 Mar 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 9; 2026: 4.

15 papers.

Sort:

  1. 2025SGD with Adaptive Preconditioning: Unified Analysis and Momentum Accelerationcited by 12
  2. 2025Towards hyperparameter-free optimization with differential privacycited by 7
  3. 2025Nesterov Finds GRAAL: Optimal and Adaptive Gradient Method for Convex Optimizationcited by 5
  4. 2025Gradient-Variation Online Adaptivity for Accelerated Optimization with H"older Smoothnesscited by 5
  5. 2025Parameter-free Algorithms for the Stochastically Extended Adversarial Modelcited by 3
  6. 2024Towards Simple and Provable Parameter-Free Adaptive Gradient Methodscited by 2
  7. 2025AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descentcited by 2
  8. 2024Faster Stochastic Optimization with Arbitrary Delays via Asynchronous Mini-Batchingcited by 1
  9. 2025The Sample Complexity of Parameter-Free Stochastic Convex Optimizationcited by 1
  10. 2025Optimistic Online-to-Batch Conversions for Accelerated Convergence and Universalitycited by 1
  11. 2026Randomized Feasibility Methods for Constrained Optimization with Adaptive Step Sizescited by 1
  12. 2025AutoGD: Automatic Learning Rate Selection for Gradient Descentcited by 0
3 more
  1. 2026Stochastic Auto-conditioned Fast Gradient Methods with Optimal Ratescited by 0
  2. 2026Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuningcited by 0
  3. 2026Clipping the Price of Adaptivity at the Tailcited by 0