# Tuning Learning Rates with the Cumulative-Learning Constant

arXiv: https://arxiv.org/abs/2505.13457 · HTML: https://mlsanity.com/paper/2505.13457/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 5 references that are in the map.
- First posted on arXiv 30 Apr 2025.

## Builds on (most cited first, 2 of 2)

- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [Gradient-based Hyperparameter Optimization through Reversible Learning](https://mlsanity.com/paper/1502.03492/) (2015)

## Built on by (most cited first, 1 of 1)

- [Cumulative Learning Rate Adaptation: Revisiting Path-Based Schedules for SGD and Adam](https://mlsanity.com/paper/2508.05408/) (2025)

## Abstract (arXiv)

This paper introduces a novel method for optimizing learning rates in machine learning. A previously unrecognized proportionality between learning rates and dataset sizes is discovered, providing valuable insights into how dataset scale influences training dynamics. Additionally, a cumulative learning constant is identified, offering a framework for designing and optimizing advanced learning rate schedules. These findings have the potential to enhance training efficiency and performance across a wide range of machine learning applications.