Unveiling the Reasoning Process of Large Language Models
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Large language models often reason beyond surface tokens, but the internal stage at which token-level information becomes abstract relational structure remains unclear. We investigate this question by analyzing how attention heads and layers transform information during autoregressive reasoning. Across mathematical and symbolic reasoning tasks, we observe a consistent layer-wise division of labor: outer layers mainly preserve and route input-related features, whereas middle layers reorganize them into more transferable rule-level representations. This interpretation is supported by representation geometry: middle-layer states occupy lower-dimensional manifolds and show stronger alignment across disjoint vocabularies that instantiate the same symbolic rules. It is further supported by causal interventions: removing middle-layer components identified by our interaction-based criterion produces substantially larger downstream changes and accuracy drops than removing components from other regions or at random. Together, these results suggest that abstract reasoning is not uniformly distributed across transformer layers, but is preferentially formed in a middle-layer computation stage that converts token-level information into reusable relational structure.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 10 of its 37 references that are in the map. First posted on arXiv 31 Mar 2026.
Builds on
10 of 37 references.
Closest ancestors · 3
Same direction, newest first
- 2025Qwen3 Technical Report14 May
- 2025Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words9 Jan
- 2024DeepSeek-V3 Technical Report27 Dec
Other references · 7
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2025Gemma 3 Technical Reportcited by 1,686
- 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 138
- 2025Weight-sparse transformers have interpretable circuitscited by 40
- 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 18
- 2026A Brain-like Synergistic Core in LLMs Drives Behaviour and Learningcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Towards a Mechanistic Understanding of Propositional Logical Reasoning in Large Language Models
- 2025Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models
- 2026Reasoning as State Transition: A Representational Analysis of Reasoning Evolution in Large Language Models
- 2026Reasoning emerges from constrained inference manifolds in large language models
- 2026Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs
- 2026Abstract Activation Spaces for Content-Invariant Reasoning in Large Language Models
- 2023Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners
- 2025The Geometry of Reasoning: Flowing Logics in Representation Space