mlsanityData to 14 Oct 2026

Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failures

arXiv 2601.19928 ↗First posted 11 Jan 2026cs.CLcited by 6

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning (RL) has catalyzed the emergence of Large Reasoning Models (LRMs) that have pushed reasoning capabilities to new heights. While their performance has garnered significant excitement, exploring the internal mechanisms driving these behaviors has become an equally critical research frontier. This paper provides a comprehensive survey of the mechanistic understanding of LRMs, organizing recent findings into three core dimensions: 1) training dynamics, 2) reasoning mechanisms, and 3) unintended behaviors. By synthesizing these insights, we aim to bridge the gap between black-box performance and mechanistic transparency. Finally, we discuss under-explored challenges to outline a roadmap for future mechanistic studies, including the need for applied interpretability, improved methodologies, and a unified theoretical framework.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 119 of its 128 references that are in the map. First posted on arXiv 11 Jan 2026.

Builds on

119 of 128 references.

Closest ancestors · 32

Same direction, newest first

  1. 2025Trust-Region Adaptive Policy Optimization19 Dec
  2. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models8 Dec
  3. 2025SCOPE-RL: Stable and Quantitative Control of Policy Entropy in RL Post-Training9 Oct
  4. 2025The Debate on RLVR Reasoning Capability Boundary: Shrinkage, Expansion, or Both? A Two-Stage Dynamic View5 Oct
  5. 2025The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models2 Oct
  6. 2025On Predictability of Reinforcement Learning Dynamics for Large Language Models1 Oct
  7. 2025Attention as a Compass: Efficient Exploration for Process-Supervised RL in Reasoning Models30 Sep
  8. 2025Rethinking Entropy Regularization in Large Reasoning Models29 Sep
  9. 2025Semantic-Space Exploration and Exploitation in RLVR for LLM Reasoning28 Sep
  10. 2025Emergent Slow Thinking in LLMs as Inverse Tree Freezing28 Sep
  11. 2025RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs25 Sep
  12. 2025A Survey of Reinforcement Learning for Large Reasoning Models10 Sep
20 more
  1. 2025RL Fine-Tuning Heals OOD Forgetting in SFT8 Sep
  2. 2025Beyond Two-Stage Training: Cooperative SFT and RL for LLM Reasoning8 Sep
  3. 2025Outcome-based Exploration for LLM Reasoning8 Sep
  4. 2025Emergent Hierarchical Reasoning in LLMs through Reinforcement Learning3 Sep
  5. 2025CURE: Critical-Token-Guided Re-Concatenation for Entropy-Collapse Prevention14 Aug
  6. 2025Teaching Models to Verbalize Reward Hacking in Chain-of-Thought Reasoning28 Jun
  7. 2025OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling25 Jun
  8. 2025SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning24 Jun
  9. 2025Reasoning with Exploration: An Entropy Perspective17 Jun
  10. 2025Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questions9 Jun
  11. 2025Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning5 Jun
  12. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models30 May
  13. 2025How Much Backtracking is Enough? Exploring the Interplay of SFT and RL in Enhancing LLM Reasoning30 May
  14. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models28 May
  15. 2025UFT: Unifying Supervised and Reinforcement Fine-Tuning22 May
  16. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  17. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  18. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs3 Mar
  19. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  20. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov

Other references · 87

  1. 2020Scaling Laws for Neural Language Modelscited by 6,586
  2. 2024OpenAI o1 System Cardcited by 1,244
  3. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  4. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
  5. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  6. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  7. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  8. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  9. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 302
  10. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  11. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 249
  12. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithfulcited by 170
38 more
  1. 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMscited by 163
  2. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 159
  3. 2025The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Taskscited by 152
  4. 2025AdaptThink: Reasoning Models Can Learn When to Thinkcited by 147
  5. 2025Thought Anchors: Which LLM Reasoning Steps Matter?cited by 129
  6. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoningcited by 119
  7. 2025Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMscited by 117
  8. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinkingcited by 99
  9. 2025The Invisible Leash: Why RLVR May or May Not Escape Its Origincited by 97
  10. 2025DeepSeek-R1 Thoughtology: Let's think about LLM Reasoningcited by 92
  11. 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1cited by 91
  12. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 91
  13. 2025Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoningcited by 85
  14. 2025Are DeepSeek R1 And Other Reasoning Models More Faithful?cited by 72
  15. 2025Understanding Reasoning in Thinking Language Models via Steering Vectorscited by 72
  16. 2025SEAL: Steerable Reasoning Calibration of Large Language Models for Freecited by 68
  17. 2025Are Reasoning Models More Prone to Hallucination?cited by 64
  18. 2025Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?cited by 58
  19. 2025Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Modelscited by 53
  20. 2025Understanding Aha Moments: from External Observations to Internal Mechanismscited by 50
  21. 2025Inverse Scaling in Test-Time Computecited by 50
  22. 2025I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoderscited by 45
  23. 2025From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by Composing Old Onescited by 42
  24. 2025Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokenscited by 41
  25. 2025The Hallucination Tax of Reinforcement Finetuningcited by 40
  26. 2025On Entropy Control in LLM-RL Algorithmscited by 40
  27. 2025Towards a Unified View of Large Language Model Post-Trainingcited by 36
  28. 2025Tracing the Representation Geometry of Language Models from Pretraining to Post-trainingcited by 35
  29. 2025Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspectivecited by 34
  30. 2025Mitigating Overthinking in Large Reasoning Models via Manifold Steeringcited by 33
  31. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 30
  32. 2025Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Propertiescited by 29
  33. 2025Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findingscited by 27
  34. 2025Let LRMs Break Free from Overthinking via Self-Braking Tuningcited by 27
  35. 2025Base Models Know How to Reason, Thinking Models Learn Whencited by 27
  36. 2025Towards Understanding Distilled Reasoning Models: A Representational Approachcited by 26
  37. 2025Reasoning-Finetuning Repurposes Latent Representations in Base Modelscited by 26
  38. 2025Attention Illuminates LLM Reasoning: The Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimizationcited by 25

50 most cited of 87 All 87 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Why Steering Works: Toward a Unified View of Language Model Parameter Dynamicscited by 9
  2. 2026Quantifying Faithful Confidence Expression in Large Reasoning Modelscited by 2
  3. 2026PRISM: A Dual View of LLM Reasoning through Semantic Flow and Latent Computationcited by 1
  4. 2026LLM Reasoning Is Latent, Not the Chain of Thoughtcited by 0
  5. 2026The Cognitive Penalty: Ablating System 1 and System 2 Reasoning in Edge-Native SLMs for Decentralized Consensuscited by 0
  6. 2026Hallucination Across the Reasoning Lifecycle: Interface Visibility, Causal Evidence, and Release Control in Large Reasoning Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
  2. 2025Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection
  3. 2026Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
  4. 2025Beyond 'Aha!': Toward Systematic Meta-Abilities Alignment in Large Reasoning Models
  5. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligence
  6. 2025Reasoning Under 1 Billion: Memory-Augmented Reinforcement Learning for Large Language Models
  7. 2025Can Large Reasoning Models Self-Train?
  8. 2025Training Language Models to Reason Efficiently