# Disentangling Recall and Reasoning in Transformer Models through Layer-wise Attention and Activation Analysis

arXiv: https://arxiv.org/abs/2510.03366 · HTML: https://mlsanity.com/paper/2510.03366/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 13 of its 35 references that are in the map.
- First posted on arXiv 3 Oct 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)
- Mechanistic interpretability / [Circuits and mechanistic analysis](https://mlsanity.com/mechinterp/circuits-and-mechanistic-analysis/)

## Builds on (most cited first, 13 of 13)

- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [A Survey on Neural Network Interpretability](https://mlsanity.com/paper/2012.14261/) (2020)
- [Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability](https://mlsanity.com/paper/2301.04709/) (2023)
- [Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models](https://mlsanity.com/paper/2406.10162/) (2024)
- [LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages](https://mlsanity.com/paper/2406.06196/) (2024)
- [Linguini: A benchmark for language-agnostic linguistic reasoning](https://mlsanity.com/paper/2409.12126/) (2024)
- [PuzzLing Machines: A Challenge on Learning From Small Data](https://mlsanity.com/paper/2004.13161/) (2020)
- [UNVEILING: What Makes Linguistics Olympiad Puzzles Tricky for LLMs?](https://mlsanity.com/paper/2508.11260/) (2025)

## Built on by (most cited first, 4 of 4)

- [RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering](https://mlsanity.com/paper/2601.09269/) (2026)
- [When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions](https://mlsanity.com/paper/2605.22873/) (2026)
- [EyeLayer: Integrating Human Attention Patterns into LLM-Based Code Summarization](https://mlsanity.com/paper/2602.22368/) (2026)
- [Towards Explainability of SLMs by investigating Token Level Activation](https://mlsanity.com/paper/2605.22377/) (2026)

## Abstract (arXiv)

Transformer-based language models excel at both recall (retrieving memorized facts) and reasoning (performing multi-step inference), but whether these abilities rely on distinct internal mechanisms remains unclear. Distinguishing recall from reasoning is crucial for predicting model generalization, designing targeted evaluations, and building safer interventions that affect one ability without disrupting the other.We approach this question through mechanistic interpretability, using controlled datasets of synthetic linguistic puzzles to probe transformer models at the layer, head, and neuron level. Our pipeline combines activation patching and structured ablations to causally measure component contributions to each task type. Across two model families (Qwen and LLaMA), we find that interventions on distinct layers and attention heads lead to selective impairments: disabling identified "recall circuits" reduces fact-retrieval accuracy by up to 15% while leaving reasoning intact, whereas disabling "reasoning circuits" reduces multi-step inference by a comparable margin. At the neuron level, we observe task-specific firing patterns, though these effects are less robust, consistent with neuronal polysemanticity.Our results provide the first causal evidence that recall and reasoning rely on separable but interacting circuits in transformer models. These findings advance mechanistic interpretability by linking circuit-level structure to functional specialization and demonstrate how controlled datasets and causal interventions can yield mechanistic insights into model cognition, informing safer deployment of large language models.