mlsanityData to 14 Oct 2026

Select and Improve: Understanding the Mechanics of Post-Training for Reasoning

arXiv 2606.13125 ↗First posted 11 Jun 2026cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning has rapidly emerged as a key component in the training of reasoning and coding models, yet it remains poorly understood from a mechanistic perspective. We study how and through what underlying processes capabilities are acquired or enhanced via reinforcement learning post-training. Our analysis, based on controlled math reasoning experiments with Qwen-2.5-1.5B, reveals two core mechanisms: strategy selection and strategy improvement. Our results highlight the role of SFT data and reinforcement learning data in activating these mechanisms, in particular showing how supervising the model on diverse reasoning strategies can enable strategy selection and how increasing difficulty in reinforcement learning data can enable strategy improvement. Taken together, our results provide mechanistic insight into RL training and suggest practical interventions to continue scaling reasoning capabilities.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 18 of its 19 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

18 of 19 references.

References · 18

  1. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  2. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  3. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 559
  4. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 485
  5. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 289
  6. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 209
  7. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Modelscited by 194
  8. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretrainingcited by 129
  9. 2024Self-Improvement in Language Models: The Sharpening Mechanismcited by 104
  10. 2025The Invisible Leash: Why RLVR May or May Not Escape Its Origincited by 97
  11. 2025Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpeningcited by 80
  12. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Modelscited by 69
6 more
  1. 2025OMEGA: Can LLMs Reason Outside the Box in Math? Evaluating Exploratory, Compositional, and Transformative Generalizationcited by 50
  2. 2025From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by Composing Old Onescited by 42
  3. 2025RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMscited by 34
  4. 2025RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?cited by 23
  5. 2025Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selectioncited by 12
  6. 2025How Reinforcement Learning After Next-Token Prediction Facilitates Learningcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026The Model Knows Another Way: Strategy Switching for Effective RLVR Explorationcited by 1
  2. 2026SFT Leaves a Reproducible Signature in Activation Geometry: Calibrating Post-Training Interpretability Against Run-to-Run Noisecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Rethinking Expert Trajectory Utilization in LLM Post-training for Mathematical Reasoning
  2. 2026How Reasoning Evolves from Post-Training Data: An Empirical Study Using Chess
  3. 2026How RL Reshapes LLM Reasoning: Transferability, Coverage, and Scaling Laws
  4. 2025Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning
  5. 2026A Primer in Post-Training Reasoning Data: What We Know About How It Works
  6. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failures
  7. 2026Probing the Origins of Reasoning Performance: Representational Quality for Mathematical Problem-Solving in RL vs. SFT Fine-Tuned Models
  8. 2026Demystifying Reinforcement Learning Post-Training of Language Models