mlsanityData to 14 Oct 2026

Tracking Equivalent Mechanistic Interpretations Across Neural Networks

arXiv 2603.30002 ↗First posted 31 Mar 2026cs.LG cs.CLcited by 0

On the map: Mechanistic interpretability → Circuits and mechanistic analysis

Abstract (arXiv)

Mechanistic interpretability (MI) is an emerging framework for interpreting neural networks. Given a task and model, MI aims to discover a succinct algorithmic process, an interpretation, that explains the model's decision process on that task. However, MI is difficult to scale and generalize. This stems in part from two key challenges: there is no precise notion of a valid interpretation; and, generating interpretations is often an ad hoc process. In this paper, we address these challenges by defining and studying the problem of interpretive equivalence: determining whether two different models share a common interpretation, without requiring an explicit description of what that interpretation is. At the core of our approach, we propose and formalize the principle that two interpretations of a model are equivalent if all of their possible implementations are also equivalent. We develop an algorithm to estimate interpretive equivalence and case study its use on Transformer-based models. To analyze our algorithm, we introduce necessary and sufficient conditions for interpretive equivalence based on models' representation similarity. We provide guarantees that simultaneously relate a model's algorithmic interpretations, circuits, and representations. Our framework lays a foundation for the development of more rigorous evaluation methods of MI and automated, generalizable interpretation discovery methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 14 of its 79 references that are in the map. First posted on arXiv 31 Mar 2026.

Builds on

14 of 79 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?11 Jul
  2. 2023Attribution Patching Outperforms Automated Circuit Discovery16 Oct
  3. 2023Towards Automated Circuit Discovery for Mechanistic Interpretability28 Apr

Other references · 11

  1. 2022Toy Models of Superpositioncited by 971
  2. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  3. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  4. 2024How to use and interpret activation patchingcited by 214
  5. 2023Localizing Model Behavior with Path Patchingcited by 195
  6. 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 156
  7. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 149
  8. 2023Getting aligned on representational alignmentcited by 143
  9. 2017Causal Consistency of Structural Equation Modelscited by 118
  10. 2022Abstraction between Structural Causal Models: A Review of Definitions and Propertiescited by 15
  11. 2024TracrBench: Generating Interpretability Testbeds with Large Language Modelscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models
  2. 2025Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networks
  3. 2025Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?
  4. 2024Validating Mechanistic Interpretations: An Axiomatic Approach
  5. 2026Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning
  6. 2025Combining Causal Models for More Accurate Abstractions of Neural Networks
  7. 2025Mechanistic Interpretability Needs Philosophy
  8. 2024Challenges in Mechanistically Interpreting Model Representations