# The Cognitive Revolution in Interpretability: From Explaining Behavior to Interpreting Representations and Algorithms

arXiv: https://arxiv.org/abs/2408.05859 · HTML: https://mlsanity.com/paper/2408.05859/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Aug 2024.

## Built on by (most cited first, 13 of 13)

- [Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?](https://mlsanity.com/paper/2502.20914/) (2025)
- [Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2601.14004/) (2026)
- [Thinking beyond the anthropomorphic paradigm benefits LLM research](https://mlsanity.com/paper/2502.09192/) (2025)
- [The Dead Salmons of AI Interpretability](https://mlsanity.com/paper/2512.18792/) (2025)
- [How Reliable are Causal Probing Interventions?](https://mlsanity.com/paper/2408.15510/) (2024)
- [Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networks](https://mlsanity.com/paper/2511.19265/) (2025)
- [Focus On This, Not That! Steering LLMs with Adaptive Feature Specification](https://mlsanity.com/paper/2410.22944/) (2024)
- [Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence](https://mlsanity.com/paper/2608.12036/) (2026)
- [Cognitive Maps in Language Models: A Mechanistic Analysis of Spatial Planning](https://mlsanity.com/paper/2511.13371/) (2025)
- [Mechanistic Interpretability of Brain-to-Speech Models Across Speech Modes](https://mlsanity.com/paper/2602.01247/) (2026)
- [Seeing Through Risk: A Symbolic Approximation of Prospect Theory](https://mlsanity.com/paper/2504.14448/) (2025)
- [Explaining the Explainer: Understanding the Inner Workings of Transformer-based Symbolic Regression Models](https://mlsanity.com/paper/2602.03506/) (2026)
- [Mechanistically Interpretable Neural Encoding Reveals Fine-Grained Functional Selectivity in Human Visual Cortex](https://mlsanity.com/paper/2605.16468/) (2026)

## Abstract (arXiv)

Artificial neural networks have long been understood as "black boxes": though we know their computation graphs and learned parameters, the knowledge encoded by these weights and functions they perform are not inherently interpretable. As such, from the early days of deep learning, there have been efforts to explain these models' behavior and understand them internally; and recently, mechanistic interpretability (MI) has emerged as a distinct research area studying the features and implicit algorithms learned by foundation models such as large language models. In this work, we aim to ground MI in the context of cognitive science, which has long struggled with analogous questions in studying and explaining the behavior of "black box" intelligent systems like the human brain. We leverage several important ideas and developments in the history of cognitive science to disentangle divergent objectives in MI and indicate a clear path forward. First, we argue that current methods are ripe to facilitate a transition in deep learning interpretation echoing the "cognitive revolution" in 20th-century psychology that shifted the study of human psychology from pure behaviorism toward mental representations and processing. Second, we propose a taxonomy mirroring key parallels in computational neuroscience to describe two broad categories of MI research, semantic interpretation (what latent representations are learned and used) and algorithmic interpretation (what operations are performed over representations) to elucidate their divergent goals and objects of study. Finally, we elaborate the parallels and distinctions between various approaches in both categories, analyze the respective strengths and weaknesses of representative works, clarify underlying assumptions, outline key challenges, and discuss the possibility of unifying these modes of interpretation under a common framework.