# Reinforcement Learning for Learning Rate Control

arXiv: https://arxiv.org/abs/1705.11159 · HTML: https://mlsanity.com/paper/1705.11159/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2017.

## Built on by (most cited first, 16 of 16)

- [Learning to Learn with Generative Models of Neural Network Checkpoints](https://mlsanity.com/paper/2209.12892/) (2022)
- [Automated Dynamic Algorithm Configuration](https://mlsanity.com/paper/2205.13881/) (2022)
- [Practical tradeoffs between memory, compute, and performance in learned optimizers](https://mlsanity.com/paper/2203.11860/) (2022)
- [A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases](https://mlsanity.com/paper/2209.11208/) (2022)
- [A Generalizable Approach to Learning Optimizers](https://mlsanity.com/paper/2106.00958/) (2021)
- [FaiR-IoT: Fairness-aware Human-in-the-Loop Reinforcement Learning for Harnessing Human Variability in Personalized IoT](https://mlsanity.com/paper/2103.16033/) (2021)
- [Learning to Optimize for Reinforcement Learning](https://mlsanity.com/paper/2302.01470/) (2023)
- [Automated Reinforcement Learning: An Overview](https://mlsanity.com/paper/2201.05000/) (2022)
- [Narrowing the Focus: Learned Optimizers for Pretrained Models](https://mlsanity.com/paper/2408.09310/) (2024)
- [Efficient Meta Subspace Optimization](https://mlsanity.com/paper/2110.14920/) (2021)
- [Unbiased and Efficient Self-Supervised Incremental Contrastive Learning](https://mlsanity.com/paper/2301.12104/) (2023)
- [OLR-WA: Online Weighted Average Linear Regression in Multivariate Data Streams](https://mlsanity.com/paper/2512.14892/) (2025)
- [Hyper-parameter optimization based on soft actor critic and hierarchical mixture regularization](https://mlsanity.com/paper/2112.04084/) (2021)
- [Stepsize Learning for Policy Gradient Methods in Contextual Markov Decision Processes](https://mlsanity.com/paper/2306.07741/) (2023)
- [Dynamic Estimation of Learning Rates Using a Non-Linear Autoregressive Model](https://mlsanity.com/paper/2410.09943/) (2024)
- [SOLAR: A State-Driven Online Learning Rate Scheduler for LLM Pretraining](https://mlsanity.com/paper/2609.34681/) (2026)

## Abstract (arXiv)

Stochastic gradient descent (SGD), which updates the model parameters by adding a local gradient times a learning rate at each step, is widely used in model training of machine learning algorithms such as neural networks. It is observed that the models trained by SGD are sensitive to learning rates and good learning rates are problem specific. We propose an algorithm to automatically learn learning rates using neural network based actor-critic methods from deep reinforcement learning (RL).In particular, we train a policy network called actor to decide the learning rate at each step during training, and a value network called critic to give feedback about quality of the decision (e.g., the goodness of the learning rate outputted by the actor) that the actor made. The introduction of auxiliary actor and critic networks helps the main network achieve better performance. Experiments on different datasets and network architectures show that our approach leads to better convergence of SGD than human-designed competitors.