mlsanityData to 13 Oct 2026

Better Parameter-free Stochastic Optimization with ODE Updates for Coin-Betting

arXiv 2006.07507 ↗First posted 12 Jun 2020cs.LG stat.MLcited by 19

Not on the map. Why?

Abstract (arXiv)

Parameter-free stochastic gradient descent (PFSGD) algorithms do not require setting learning rates while achieving optimal theoretical performance. In practical applications, however, there remains an empirical gap between tuned stochastic gradient descent (SGD) and PFSGD. In this paper, we close the empirical gap with a new parameter-free algorithm based on continuous-time Coin-Betting on truncated models. The new update is derived through the solution of an Ordinary Differential Equation (ODE) and solved in a closed form. We show empirically that this new parameter-free algorithm outperforms algorithms with the "best default" learning rates and almost matches the performance of finely tuned baselines without anything to tune.

Key facts. Cited by 19 papers in the map (data through 13 Oct 2026). First posted on arXiv 12 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 5; 2023: 6; 2024: 5; 2025: 2; 2026: 1.

19 papers.

Sort:

By area:

  1. 2023DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedulecited by 85
  2. 2022Making SGD Parameter-Freecited by 50
  3. 2022PDE-Based Optimal Strategy for Unconstrained Online Learningcited by 27
  4. 2023Auditing Fairness by Bettingcited by 24
  5. 2024How Free is Parameter-Free Stochastic Optimization?cited by 12
  6. 2023Tuning-Free Maximum Likelihood Training of Latent Variable Models via Coin Bettingcited by 11
  7. 2024The Price of Adaptivity in Stochastic Convex Optimizationcited by 11
  8. 2022Optimal Comparator Adaptive Online Learning with Switching Costcited by 9
  9. 2022Implicit Parameter-free Online Learning with Truncated Linear Modelscited by 8
  10. 2023Coin Sampling: Gradient-Based Bayesian Inference without Learning Ratescited by 8
  11. 2024Adaptive Variance Reduction for Stochastic Optimization under Weaker Assumptionscited by 5
  12. 2023Generalized Implicit Follow-The-Regularized-Leadercited by 3
7 more
  1. 2024Towards Stability of Parameter-Free Optimizationcited by 1
  2. 2024Towards Simple and Provable Parameter-Free Adaptive Gradient Methodscited by 1
  3. 2025The Sample Complexity of Parameter-Free Stochastic Convex Optimizationcited by 1
  4. 2022Learning to Accelerate by the Methods of Step-size Planningcited by 0
  5. 2023Locally Optimal Descent for Dynamic Stepsize Schedulingcited by 0
  6. 2025Reasoning without Regretcited by 0
  7. 2026Clipping the Price of Adaptivity at the Tailcited by 0