Evaluating Explanations: An Explanatory Virtues Framework for Mechanistic Interpretability -- The Strange Science Part I.ii
Not on the map. Why?
Abstract (arXiv)
Mechanistic Interpretability (MI) aims to understand neural networks through causal explanations. Though MI has many explanation-generating methods, progress has been limited by the lack of a universal approach to evaluating explanations. Here we analyse the fundamental question "What makes a good explanation?" We introduce a pluralist Explanatory Virtues Framework drawing on four perspectives from the Philosophy of Science - the Bayesian, Kuhnian, Deutschian, and Nomological - to systematically evaluate and improve explanations in MI. We find that Compact Proofs consider many explanatory virtues and are hence a promising approach. Fruitful research directions implied by our framework include (1) clearly defining explanatory simplicity, (2) focusing on unifying explanations and (3) deriving universal principles for neural networks. Improved MI methods enhance our ability to monitor, predict, and steer AI systems.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 37 of its 91 references that are in the map. First posted on arXiv 2 May 2025.
Builds on
37 of 91 references.
References · 37
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
- 2024Scaling and evaluating sparse autoencoderscited by 530
- 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
- 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
- 2021Causal Abstractions of Neural Networkscited by 401
- 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 264
- 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
- 2024Foundational Challenges in Assuring Alignment and Safety of Large Language Modelscited by 193
- 2020A Theory of Usable Information Under Computational Constraintscited by 153
- 2023A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operationscited by 141
25 more
- 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 118
- 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 106
- 2018Abstracting Causal Modelscited by 104
- 2025Sparse Autoencoders Do Not Find Canonical Units of Analysiscited by 80
- 2023Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patchingcited by 75
- 2024Transformers represent belief state geometry in their residual streamcited by 73
- 2024Identifying Functionally Important Features with End-to-End Sparse Dictionary Learningcited by 64
- 2025International AI Safety Reportcited by 64
- 2024Escalation Risks from Language Models in Military and Diplomatic Decision-Makingcited by 61
- 2022Mechanistic Mode Connectivitycited by 50
- 2024Hypothesis Testing the Circuit Hypothesis in LLMscited by 29
- 2025Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decompositioncited by 25
- 2023Interpretability Illusions in the Generalization of Simplified Modelscited by 24
- 2024InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniquescited by 24
- 2021Clusterability in Neural Networkscited by 23
- 2024Local vs. Global Interpretability: A Computational Complexity Perspectivecited by 23
- 2023Grokking Group Multiplication with Cosetscited by 22
- 2024Interpretability as Compression: Reconsidering SAE Explanations of Neural Activations with MDL-SAEscited by 22
- 2024Compact Proofs of Model Performance via Mechanistic Interpretabilitycited by 17
- 2024The Computational Complexity of Circuit Discovery for Inner Interpretabilitycited by 15
- 2025Propositional Interpretability in Artificial Intelligencecited by 11
- 2024Auditing Local Explanations is Hardcited by 8
- 2020From Probability to Consilience: How Explanatory Values Implement Bayesian Reasoningcited by 7
- 2025A Mathematical Philosophy of Explanations in Mechanistic Interpretability -- The Strange Science Part I.icited by 6
- 2024TracrBench: Generating Interpretability Testbeds with Large Language Modelscited by 4
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.