Deep Learning Models Also Recall Features
On the map: Mechanistic interpretability → Circuits and mechanistic analysis
Abstract (arXiv)
Recent work in mechanistic interpretability has studied how large language models recall facts stored in their weights. This paper argues that factual recall points to something broader: a general kind of operation in deep learning models, which I call feature recall. The core observation is that a linear projection can be read as retrieving stored information scaled by input activations. I define feature recall, show it applies across architectures, and contrast it with the established paradigm of feature combination. I also consider how cases of feature recall might be mechanistically identified. The account gives philosophers a new conceptual tool for understanding deep learning, and points to empirical directions for mechanistic interpretability research.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 8 of its 19 references that are in the map. First posted on arXiv 21 Aug 2026.
Builds on
8 of 19 references.
Closest ancestors · 1
Same direction, newest first
Other references · 7
- 2022Toy Models of Superpositioncited by 971
- 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
- 2026Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnetcited by 191
- 2024Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMscited by 42
- 2024Standards for Belief Representations in LLMscited by 30
- 2025How Do LLMs Use Their Depth?cited by 16
- 2025Mechanistic Indicators of Understanding in Large Language Modelscited by 8
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Provable Benefits of In-Tool Learning for Large Language Models
- 2024The Cognitive Revolution in Interpretability: From Explaining Behavior to Interpreting Representations and Algorithms
- 2026Putting a Face to Forgetting: Continual Learning meets Mechanistic Interpretability
- 2026There Will Be a Scientific Theory of Deep Learning
- 2025A Mathematical Philosophy of Explanations in Mechanistic Interpretability -- The Strange Science Part I.i
- 2024Challenges in Mechanistically Interpreting Model Representations
- 2025Test-time regression: a unifying framework for designing sequence models with associative memory
- 2026Identifying Temporal Features within Transcoders for Time Sensitive Factual Recall