Make Mechanistic Interpretability Auditable: A Call to Develop Guidelines via Continuous Collaborative Reviewing
On the map: Mechanistic interpretability → Circuits and mechanistic analysis
Abstract (arXiv)
While mechanistic interpretability (MI) has produced important insights into neural network internals, the field has yet to establish a standardized system to audit experiments. As such, many of its findings remain underutilized in safety-critical applications such as medical AI and autonomous systems, as stakeholders cannot certify their validity. Recent work demonstrates this concretely: two papers found conflicting conclusions for the same behavior, and a third study revealed that both were partially correct but incomparable due to methodological inconsistencies. Without standardized auditing, such ambiguities hinder adoption in high-stakes contexts requiring strong correctness guarantees. We call for the MI community to work towards developing a novel reviewing system that complements peer review via: (1) Continuous reviewing supported by a Collaborative Reviewing Platform where meta-science results and discussions (such as critiques, negative results, post-hoc extensions, reproductions, replications, and partial results) that fit outside of papers are organized and discussed, allowing for comments and revisions to be made at any time (2) Generalizing good practices found on this platform into expert-verified guidelines and protocols to improve auditing efficiency, and (3) Source-based auditing systems that track arguments which claims depend on. This position paper encourages constructive debate over the necessity, design and implementation of such a framework, providing early concrete examples to help catalyze these dialogues. Overall, we propose that auditing MI itself is essential for its application in AI safety, industry, and governance.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 31 of its 80 references that are in the map. First posted on arXiv 24 Apr 2026.
Builds on
31 of 80 references.
Closest ancestors · 9
Same direction, newest first
- 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?11 Jul
- 2025Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?28 Feb
- 2024Hypothesis Testing the Circuit Hypothesis in LLMs16 Oct
- 2024Towards a unified and verified understanding of group-operation networks9 Oct
- 2024InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniques19 Jul
- 2024Mechanistic Interpretability for AI Safety -- A Review22 Apr
- 2023A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations6 Feb
- 2023Progress measures for grokking via mechanistic interpretability12 Jan
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small1 Nov
Other references · 22
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2018Sanity Checks for Saliency Mapscited by 1,156
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
- 2019Neural Probabilistic Logic Programming in DeepProbLogcited by 394
- 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 391
- 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
- 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Proverscited by 168
- 2017End-to-End Differentiable Provingcited by 161
- 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 160
- 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 156
- 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 143
- 2023Tracr: Compiled Transformers as a Laboratory for Interpretabilitycited by 104
10 more
- 2024LLM Circuit Analyses Are Consistent Across Training and Scalecited by 63
- 2024Reproducibility in Machine Learning-based Research: Overview, Barriers and Driverscited by 34
- 2025Real-Time Detection of Hallucinated Entities in Long-Form Generationcited by 34
- 2023Interpretability Illusions in the Generalization of Simplified Modelscited by 25
- 2025Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEscited by 14
- 2025The Dead Salmons of AI Interpretabilitycited by 12
- 2024Position: Embracing Negative Results in Machine Learningcited by 6
- 2020ReproducedPapers.org: Openly teaching and structuring machine learning reproducibilitycited by 5
- 2024DILA: Dictionary Label Attention for Mechanistic Interpretability in High-dimensional Multi-label Medical Coding Predictioncited by 4
- 2025Causal Bayesian Networks for Data-driven Safety Analysis of Complex Systemscited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Mechanistic Interpretability Needs Philosophy
- 2025Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networks
- 2025Open Problems in Mechanistic Interpretability
- 2026Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning
- 2024Mechanistic?
- 2024Challenges in Mechanistically Interpreting Model Representations
- 2026Rethinking Circuit Evaluation: Do Circuits Explain Model Errors?
- 2024Validating Mechanistic Interpretations: An Axiomatic Approach