mlsanityData to 7 Oct 2026

(How) Learning Rates Regulate Catastrophic Overtraining

arXiv 2604.13627 ↗First posted 15 Apr 2026cs.LG cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Supervised fine-tuning (SFT) is a common first stage of LLM post-training, teaching the model to follow instructions and shaping its behavior as a helpful assistant. At the same time, SFT may harm the fundamental capabilities of an LLM, particularly after long pretraining: a phenomenon known as catastrophic overtraining (Springer et al., 2025). To understand overtraining, we first investigate catastrophic forgetting in finetuning through the lens of implicit regularization of the learning rate. For models trained to the same SFT loss, we identify how the learning rate mediates optimization: finetuning with large and small steps converges to qualitatively different models. Next, we link forgetting to overtraining: learning rate decay increases the sharpness of the pretrained model, which in turn exacerbates catastrophic forgetting during SFT, leading to overtraining. Our findings paint a picture of the overtraining mechanism in LLMs and broadly contribute to the understanding of the interplay between optimization dynamics during pretraining and finetuning.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 51 of its 67 references that are in the map. First posted on arXiv 15 Apr 2026.

Builds on

51 of 67 references.

References · 51

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  7. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  8. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  9. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  10. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  11. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  12. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
38 more
  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  3. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  4. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  5. 2018On Lazy Training in Differentiable Programmingcited by 671
  6. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
  7. 2024OLMo: Accelerating the Science of Language Modelscited by 599
  8. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
  9. 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 413
  10. 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 381
  11. 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 339
  12. 2024LoRA Learns Less and Forgets Lesscited by 313
  13. 2020Kernel and Rich Regimes in Overparametrized Modelscited by 302
  14. 20242 OLMo 2 Furiouscited by 302
  15. 2020The large learning rate phase of deep learning: the catapult mechanismcited by 206
  16. 2019Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networkscited by 162
  17. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
  18. 2019Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like processcited by 123
  19. 2020Finding Universal Grammatical Relations in Multilingual BERTcited by 108
  20. 2024Revisiting Catastrophic Forgetting in Large Language Model Tuningcited by 93
  21. 2023A Modern Look at the Relationship between Sharpness and Generalizationcited by 91
  22. 2022Adaptive Gradient Methods at the Edge of Stabilitycited by 88
  23. 2022Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Modelscited by 84
  24. 2018A Walk with SGDcited by 74
  25. 2024On the Role of Attention Heads in Large Language Model Safetycited by 70
  26. 2022SGD with Large Step Sizes Learns Sparse Featurescited by 66
  27. 2024Understanding Optimization in Deep Learning with Central Flowscited by 57
  28. 2025Overtrained Language Models Are Harder to Fine-Tunecited by 57
  29. 2024Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMscited by 39
  30. 2025RL Fine-Tuning Heals OOD Forgetting in SFTcited by 27
  31. 2025SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMscited by 23
  32. 2025Upweighting Easy Samples in Fine-Tuning Mitigates Forgettingcited by 16
  33. 2025Hubble: a Model Suite to Advance the Study of LLM Memorizationcited by 16
  34. 2025Training Dynamics Impact Post-Training Quantization Robustnesscited by 13
  35. 2026Replaying pre-training data improves fine-tuningcited by 12
  36. 2026Pre-training LLM without Learning Rate Decay Enhances Supervised Fine-Tuningcited by 12
  37. 2025Context-Free Synthetic Data Mitigates Forgettingcited by 5
  38. 2026Universal Dynamics of Warmup Stable Decay: understanding WSD beyond Transformerscited by 5

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Lesscited by 3
  2. 2026Are we Merging the Right Models? Impact of Expert Training Duration on Model Merging for LLMscited by 1
  3. 2026Linger and Lose: Knowledge Collapse in Low-Bit Language Modelscited by 0
  4. 2026On-Policy or Off-Policy Learning? A Systematic Study of Distillation Dynamicscited by 0