From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Models
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Large Reasoning Models (LRMs) generate explicit reasoning traces alongside final answers, yet the extent to which these traces influence answer generation remains unclear. In this work, we conduct a three-stage investigation into the interplay between reasoning and answer generation in three distilled DeepSeek R1 models. First, through empirical evaluation, we demonstrate that including explicit reasoning consistently improves answer quality across diverse domains. Second, attention analysis reveals that answer tokens attend substantially to reasoning tokens, with certain mid-layer Reasoning-Focus Heads (RFHs) closely tracking the reasoning trajectory, including self-reflective cues. Third, we apply mechanistic interventions using activation patching to assess the dependence of answer tokens on reasoning activations. Our results show that perturbations to key reasoning tokens can reliably alter the final answers, confirming a directional and functional flow of information from reasoning to answer. These findings deepen our understanding of how LRMs leverage reasoning tokens for answer generation, highlighting the functional role of intermediate reasoning in shaping model outputs. Our data and code are publicly available at hrefhttps://aka.ms/R2A-codethis URL.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 19 of its 48 references that are in the map. First posted on arXiv 28 Sep 2025.
Builds on
19 of 48 references.
Closest ancestors · 3
Same direction, newest first
- 2025Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs30 Apr
- 2025Reasoning Models Can Be Effective Without Thinking14 Apr
- 2025The relationship between reasoning and performance in large language models--o3 (mini) thinks harder, not longer21 Feb
Other references · 16
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
- 2024OpenAI o1 System Cardcited by 1,244
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
- 2022In-context Learning and Induction Headscited by 964
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
- 2025Reasoning Models Don't Always Say What They Thinkcited by 415
- 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
- 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 232
- 2024How to use and interpret activation patchingcited by 214
- 2025Why do LLMs attend to the first token?cited by 127
- 2025DeepSeek-R1 Thoughtology: Let's think about LLM Reasoningcited by 92
- 2023On Measuring Faithfulness or Self-consistency of Natural Language Explanationscited by 73
4 more
- 2025Are DeepSeek R1 And Other Reasoning Models More Faithful?cited by 72
- 2025I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoderscited by 45
- 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
- 2025Evaluating Mathematical Reasoning Across Large Language Models: A Fine-Grained Approachcited by 8
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failurescited by 6
- 2026Plans Don't Persist: Why Context Management Is Load Bearing for LLM Agentscited by 4
- 2026Contrastive Attribution in the Wild: An Interpretability Analysis of LLM Failures on Realistic Benchmarkscited by 2
- 2026Transcoder Adapters for Reasoning-Model Diffingcited by 1
- 2026How Do Answer Tokens Read Reasoning Traces? Self-Reading Patterns in Thinking LLMs for Quantitative Reasoningcited by 1
- 2026Reachability Is Not Realization: Tracing the Sources of LLM Benchmark Gainscited by 1
- 2026Reachability Is Not Realization: Tracing the Sources of LLM Benchmark Gains4 Aug 2026
- 2026Plans Don't Persist: Why Context Management Is Load Bearing for LLM Agents22 Jun 2026
- 2026How Do Answer Tokens Read Reasoning Traces? Self-Reading Patterns in Thinking LLMs for Quantitative Reasoning21 Apr 2026
- 2026Contrastive Attribution in the Wild: An Interpretability Analysis of LLM Failures on Realistic Benchmarks20 Apr 2026
- 2026Transcoder Adapters for Reasoning-Model Diffing24 Feb 2026
- 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failures11 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs
- 2026Large Language Models Decide Early and Explain Later
- 2025Leveraging Reasoning Model Answers to Enhance Non-Reasoning Model Capability
- 2025The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Direction
- 2026Probing the Trajectories of Reasoning Traces in Large Language Models
- 2026Reasoning that Travels: Dissecting How Chain-of-Thought Transfers Across Models
- 2026Every Token Leaves a Ripple in the Stream of Thought: Eliciting Model-Internal Token Saliency for Chain-of-Thought Compression
- 2025First Try Matters: Revisiting the Role of Reflection in Reasoning Models