mlsanityData to 7 Oct 2026

SOLAR: A State-Driven Online Learning Rate Scheduler for LLM Pretraining

arXiv 2609.34681 ↗First posted 28 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Learning-rate (LR) scheduling plays a central role in large language model (LLM) pretraining, yet current practice still relies heavily on hand-crafted heuristics such as Warmup-Cosine-Decay and Warmup-Stable-Decay. Because these schedules are fixed in advance, they cannot adapt to evolving optimization dynamics. Online learned scheduling within the Learning to Optimize (L2O) framework offers a dynamic alternative, but remains brittle at LLM scale due to noisy signals, delayed feedback, and the risk of catastrophic divergence. We propose SOLAR (State-driven Online Learning rAte scheduleR), a stabilized framework for reliable online LR adaptation. SOLAR uses a base schedule as a reference and learns bounded, state-dependent residual corrections for individual parameter groups. Each correction re-anchors to the base at every step, allowing the policy to adapt the LR without relearning the warmup-decay profile. A lightweight state representation and progress-aware reward guide online learning, while a Circuit-Breaker restores training after rare unsafe actions. Across autoregressive language-model pretraining, SOLAR improves final perplexity over tuned static schedules and automatic LR tuners for dense models from 60M to 1B, AdamW and Muon, and two MoE settings up to 3B. Matched 130M controls show that adding base anchoring and action bounds improves a global PPO controller from 27.09 to 23.74 final PPL, while group-wise control reaches 22.87 on the same two seeds. A residual policy trained on a 60M proxy can also be frozen and reused at larger dense scales without target PPO updates, remaining effective across a fourfold base-LR range. These results establish SOLAR as a practical learned LR controller for LLM pretraining.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 53 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

38 of 53 references.

References · 38

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  6. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  7. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  8. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  9. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  10. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  11. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  12. 2015Cyclical Learning Rates for Training Neural Networkscited by 627
26 more
  1. 2016Learning to learn by gradient descent by gradient descentcited by 616
  2. 2024GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projectioncited by 447
  3. 2023Symbolic Discovery of Optimization Algorithmscited by 423
  4. 2025Muon is Scalable for LLM Trainingcited by 381
  5. 2017Population Based Training of Neural Networkscited by 351
  6. 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 272
  7. 2018An Empirical Model of Large-Batch Trainingcited by 268
  8. 2021Learning to Optimize: A Primer and A Benchmarkcited by 206
  9. 2016Learning to Optimizecited by 195
  10. 2024The Road Less Scheduledcited by 158
  11. 2023Prodigy: An Expeditiously Adaptive Parameter-Free Learnercited by 129
  12. 2017Online Learning Rate Adaptation with Hypergradient Descentcited by 117
  13. 2023Learning-Rate-Free Learning by D-Adaptationcited by 113
  14. 2017Learned Optimizers that Scale and Generalizecited by 109
  15. 2023Spike No More: Stabilizing the Pre-training of Large Language Modelscited by 51
  16. 2019Learning an Adaptive Learning Rate Schedulecited by 29
  17. 2025SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Trainingcited by 27
  18. 2025The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Trainingcited by 27
  19. 2023Mechanic: A Learning Rate Tunercited by 20
  20. 2017Reinforcement Learning for Learning Rate Controlcited by 16
  21. 2021AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Flycited by 14
  22. 2023Towards Constituting Mathematical Structures for Learning to Optimizecited by 13
  23. 2026Accelerating LLM Pre-Training through Flat-Direction Dynamics Enhancementcited by 8
  24. 2016Using Deep Q-Learning to Control Optimization Hyperparameterscited by 4
  25. 2024A Mathematics-Inspired Learning-to-Optimize Framework for Decentralized Optimizationcited by 3
  26. 2023Generalisable Agents for Neural Network Optimisationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Scaling and Transferability of Annealing Strategies in Large Language Model Training
  2. 2026Learning at the Right Pace: Adaptive Data Scheduling Improves LLM Reinforcement Learning
  3. 2026Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning
  4. 2023Rethinking Learning Rate Tuning in the Era of Large Language Models
  5. 2026Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning
  6. 2026Learning-Zone Energy: Online Data Selection for Efficient RL Post-Training
  7. 2025Single-stream Policy Optimization
  8. 2026Learning, Fast and Slow: Towards LLMs That Adapt Continually