Fine Until Fine-Tuned: Repeated Solutions Make Reasoning Fragile
Not on the map. Why?
Abstract (arXiv)
Recipes such as s1 and LIMO teach a model to reason with little data by showing it the same thousand or fewer worked solutions many times over. Judged when that training ends, the repetition looks harmless. But reasoning models are often trained again, and we find that repetition leaves their reasoning fragile to that next stage, even when the stage has nothing to do with reasoning. We fine-tuned Qwen3.5-9B-Base on its own correct solutions to competition math problems, either drilling a few hundred of them about eight times each or showing many more once; with the same amount of training, both solve about 95% of held-out problems. A single pass of ordinary instruction tuning leaves the once-trained model where it was, while the drilled one falls to 86.0%, and harsher later stages take it to 59.3% or below. A third model that visited the drilled problems just as often, with a new solution at every visit, was unharmed, so the damage comes from seeing the same texts again rather than from having few problems. The break recurs with a stronger model's traces, in further training runs and on other models and tasks. It is also cheap to undo: the reasoning is suppressed rather than erased, and five updates of reasoning training bring almost all of it back, as does brief training on the reasoning format with almost no mathematics. Fresh solutions prevented the damage, and so did replaying 6.25% of the original solutions in a gentler later stage, so our claim concerns later training without such replay. Sharpening alone does not explain the break, since a model sharpened three-quarters as much without repetition was unharmed. On a skill the base model could not perform within a token budget, repetition mainly cost learning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 41 references that are in the map. First posted on arXiv 27 Sep 2026.
Builds on
37 of 41 references.
References · 37
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2023Let's Verify Step by Stepcited by 3,789
- 2025s1: Simple test-time scalingcited by 1,341
- 2018Experience Replay for Continual Learningcited by 1,095
- 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
- 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
- 2025LIMO: Less is More for Reasoningcited by 498
- 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
- 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
- 2024LoRA Learns Less and Forgets Lesscited by 328
25 more
- 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 297
- 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 176
- 2022Fine-tuned Language Models are Continual Learnerscited by 156
- 2022Scaling Laws and Interpretability of Learning from Repeated Datacited by 152
- 2024Stream of Search (SoS): Learning to Search in Languagecited by 145
- 2023Understanding Catastrophic Forgetting in Language Models via Implicit Inferencecited by 144
- 2026Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMscited by 137
- 2023Mechanistically analyzing the effects of fine-tuning on procedurally defined taskscited by 115
- 2024Self-Improvement in Language Models: The Sharpening Mechanismcited by 104
- 2025Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 104
- 2024Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuningcited by 96
- 2025Retaining by Doing: The Role of On-Policy Data in Mitigating Forgettingcited by 94
- 2024How Do Large Language Models Acquire Factual Knowledge During Pretraining?cited by 90
- 2024Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearningcited by 84
- 2025Overtrained Language Models Are Harder to Fine-Tunecited by 58
- 2025Weight Ensembling Improves Reasoning in Language Modelscited by 38
- 2025Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Insteadcited by 28
- 2025Base Models Know How to Reason, Thinking Models Learn Whencited by 27
- 2025Unveiling Over-Memorization in Finetuning LLMs for Reasoning Taskscited by 11
- 2026Data Repetition Beats Data Scaling in Long-CoT Supervised Fine-Tuningcited by 8
- 2026Early Data Exposure Improves Robustness to Subsequent Fine-Tuningcited by 4
- 2026Why Do Reasoning Models Lose Coverage? The Role of Data and Forks in the Roadcited by 4
- 2026Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuningcited by 4
- 2026Internal Data Repetition Destroys Language Modelscited by 4
- 2025Retention analysis of edited knowledge after fine-tuningcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Can Large Reasoning Models Improve Accuracy on Mathematical Tasks Using Flawed Thinking?
- 2026Learn Hard Problems During RL with Reference Guided Fine-tuning
- 2024ReFT: Reasoning with Reinforced Fine-Tuning
- 2026Question Begets Question: Self-Evolving Curriculum for Reinforcement Fine-Tuning on Competition Mathematics
- 2026Capacity-Dependent Effects of Data Selection for Reasoning
- 2026The Quality-Utility Paradox: Why High-Reward Data Impairs Small Model Mathematical Reasoning
- 2026Do Instruction-Tuned Models Always Perform Better Than Base Models? Evidence from Math and Domain-Shifted Benchmarks
- 2025Unleashing the Reasoning Potential of Pre-trained LLMs by Critique Fine-Tuning on One Problem