# Mechanistic Interpretability as Statistical Estimation: A Variance Analysis

arXiv: https://arxiv.org/abs/2510.00845 · HTML: https://mlsanity.com/paper/2510.00845/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability.
- Builds on 39 of its 76 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Placed in

- Mechanistic interpretability / [Circuits and mechanistic analysis](https://mlsanity.com/mechinterp/circuits-and-mechanistic-analysis/)

## Builds on (most cited first, 30 of 39)

- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [Direct and Indirect Effects](https://mlsanity.com/paper/1301.2300/) (2013)
- [Attribution Patching Outperforms Automated Circuit Discovery](https://mlsanity.com/paper/2310.10348/) (2023)
- [How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model](https://mlsanity.com/paper/2305.00586/) (2023)
- [Transcoders Find Interpretable LLM Feature Circuits](https://mlsanity.com/paper/2406.11944/) (2024)
- [Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias](https://mlsanity.com/paper/2004.12265/) (2020)
- [Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond](https://mlsanity.com/paper/2202.06861/) (2022)
- [Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking](https://mlsanity.com/paper/2402.14811/) (2024)
- [Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms](https://mlsanity.com/paper/2403.17806/) (2024)
- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks](https://mlsanity.com/paper/2311.12786/) (2023)
- [Physics of Language Models: Part 1, Learning Hierarchical Language Structures](https://mlsanity.com/paper/2305.13673/) (2023)
- [Veridical Data Science](https://mlsanity.com/paper/1901.08152/) (2019)
- [Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark](https://mlsanity.com/paper/2305.17553/) (2023)
- [Finding Transformer Circuits with Edge Pruning](https://mlsanity.com/paper/2406.16778/) (2024)
- [MIB: A Mechanistic Interpretability Benchmark](https://mlsanity.com/paper/2504.13151/) (2025)
- [Refining Targeted Syntactic Evaluation of Language Models](https://mlsanity.com/paper/2104.09635/) (2021)
- [Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?](https://mlsanity.com/paper/2502.20914/) (2025)
- [Hypothesis Testing the Circuit Hypothesis in LLMs](https://mlsanity.com/paper/2410.13032/) (2024)

## Built on by (most cited first, 15 of 15)

- [The Dead Salmons of AI Interpretability](https://mlsanity.com/paper/2512.18792/) (2025)
- [The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching](https://mlsanity.com/paper/2606.27510/) (2026)
- [Certified Circuits: Stability Guarantees for Mechanistic Circuits](https://mlsanity.com/paper/2602.22968/) (2026)
- [Certified Interventional Fidelity: Anytime-Valid, Adaptive Evaluation of Causal Claims in Mechanistic Interpretability](https://mlsanity.com/paper/2607.08349/) (2026)
- [Many Circuits, One Mechanism: Input Variation and Evaluation Granularity in Circuit Discovery](https://mlsanity.com/paper/2606.06267/) (2026)
- [When Attribution Patching Lies: Diagnosis and a Second-Order Correction](https://mlsanity.com/paper/2606.09899/) (2026)
- [Demystifying Variance in Circuit Discovery of LLMs](https://mlsanity.com/paper/2606.16920/) (2026)
- [Circuit Claims Depend on What Is Extracted and How It Is Compared](https://mlsanity.com/paper/2607.18921/) (2026)
- [Explanation Multiplicity: Circuit-Level Interpretability Evidence Does Not Survive Defensible Analytic Variation](https://mlsanity.com/paper/2608.13754/) (2026)
- [EvoXplain: When Machine Learning Models Agree on Predictions but Disagree on Why -- Measuring Mechanistic Multiplicity Across Training Runs](https://mlsanity.com/paper/2512.22240/) (2025)
- [Induction Meets Biology: Mechanisms of Repeat Detection in Protein Language Models](https://mlsanity.com/paper/2602.23179/) (2026)
- [Necessary, Decodable and Reversible, Yet Not Transferable: A Stress Test for Attention-Head Role Claims](https://mlsanity.com/paper/2606.08292/) (2026)
- [Validating Causal Abstraction Metrics on Simulated Complex Systems](https://mlsanity.com/paper/2607.00267/) (2026)
- [Circuit Condensation: Post-Training that Concentrates a Behavior's Causal Circuit](https://mlsanity.com/paper/2608.27254/) (2026)
- [Rewired or Gated? How Instruction Tuning Shapes Knowledge-Conflict Circuits in LLMs](https://mlsanity.com/paper/2609.25602/) (2026)

## Abstract (arXiv)

Mechanistic Interpretability (MI) aims to reverse-engineer model behaviors by identifying functional sub-networks. Yet, the scientific validity of these findings depends on their stability. In this work, we argue that circuit discovery is not a standalone task but a statistical estimation problem built upon causal mediation analysis (CMA). We uncover a fundamental instability at this base layer: exact, single-input CMA scores exhibit high intrinsic variance, implying that the causal effect of a component is a volatile random variable rather than a fixed property. We then demonstrate that circuit discovery pipelines inherit this variance and further amplify it. Fast approximation methods, such as Edge Attribution Patching and its successors, introduce additional estimation noise, while aggregating these noisy scores over datasets leads to fragile structural estimates. Consequently, small perturbations in input data or hyperparameters yield vastly different circuits. We systematically decompose these sources of variance and advocate for more rigorous MI practices, prioritizing statistical robustness and routine reporting of stability metrics.