# Hypothesis Testing the Circuit Hypothesis in LLMs

arXiv: https://arxiv.org/abs/2410.13032 · HTML: https://mlsanity.com/paper/2410.13032/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability.
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 16 Oct 2024.

## Placed in

- Mechanistic interpretability / [Circuits and mechanistic analysis](https://mlsanity.com/mechinterp/circuits-and-mechanistic-analysis/)

## Builds on (most cited first, 24 of 24)

- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models](https://mlsanity.com/paper/2301.04213/) (2023)
- [Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations](https://mlsanity.com/paper/2303.02536/) (2023)
- [Attribution Patching Outperforms Automated Circuit Discovery](https://mlsanity.com/paper/2310.10348/) (2023)
- [How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model](https://mlsanity.com/paper/2305.00586/) (2023)
- [Thinking Like Transformers](https://mlsanity.com/paper/2106.06981/) (2021)
- [Compositional Explanations of Neurons](https://mlsanity.com/paper/2006.14032/) (2020)
- [The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks](https://mlsanity.com/paper/2306.17844/) (2023)
- [Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla](https://mlsanity.com/paper/2307.09458/) (2023)
- [Tracr: Compiled Transformers as a Laboratory for Interpretability](https://mlsanity.com/paper/2301.05062/) (2023)
- [Learning Transformer Programs](https://mlsanity.com/paper/2306.01128/) (2023)
- [Understanding Addition in Transformers](https://mlsanity.com/paper/2310.13121/) (2023)
- [FIND: A Function Description Benchmark for Evaluating Interpretability Methods](https://mlsanity.com/paper/2309.03886/) (2023)
- [Grokking Group Multiplication with Cosets](https://mlsanity.com/paper/2312.06581/) (2023)
- [Red Teaming Deep Neural Networks with Feature Synthesis Tools](https://mlsanity.com/paper/2302.10894/) (2023)
- [Look Before You Leap: A Universal Emergent Decomposition of Retrieval Tasks in Language Models](https://mlsanity.com/paper/2312.10091/) (2023)
- [ALMANACS: A Simulatability Benchmark for Language Model Explainability](https://mlsanity.com/paper/2312.12747/) (2023)

## Built on by (most cited first, 29 of 29)

- [MIB: A Mechanistic Interpretability Benchmark](https://mlsanity.com/paper/2504.13151/) (2025)
- [How Do LLMs Perform Two-Hop Reasoning in Context?](https://mlsanity.com/paper/2502.13913/) (2025)
- [Mechanistic Interpretability as Statistical Estimation: A Variance Analysis](https://mlsanity.com/paper/2510.00845/) (2025)
- [The Dead Salmons of AI Interpretability](https://mlsanity.com/paper/2512.18792/) (2025)
- [Are formal and functional linguistic mechanisms dissociated in language models?](https://mlsanity.com/paper/2503.11302/) (2025)
- [Formal Mechanistic Interpretability: Automated Circuit Discovery with Provable Guarantees](https://mlsanity.com/paper/2602.16823/) (2026)
- [The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching](https://mlsanity.com/paper/2606.27510/) (2026)
- [Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety](https://mlsanity.com/paper/2506.05451/) (2025)
- [Can LLMs Lie? Investigation beyond Hallucination](https://mlsanity.com/paper/2509.03518/) (2025)
- [Addressing divergent representations from causal interventions on neural networks](https://mlsanity.com/paper/2511.04638/) (2025)
- [A Mathematical Philosophy of Explanations in Mechanistic Interpretability -- The Strange Science Part I.i](https://mlsanity.com/paper/2505.00808/) (2025)
- [Circuit Stability Characterizes Language Model Generalization](https://mlsanity.com/paper/2505.24731/) (2025)
- [Certified Interventional Fidelity: Anytime-Valid, Adaptive Evaluation of Causal Claims in Mechanistic Interpretability](https://mlsanity.com/paper/2607.08349/) (2026)
- [When Attribution Patching Lies: Diagnosis and a Second-Order Correction](https://mlsanity.com/paper/2606.09899/) (2026)
- [Evaluating Explanations: An Explanatory Virtues Framework for Mechanistic Interpretability -- The Strange Science Part I.ii](https://mlsanity.com/paper/2505.01372/) (2025)
- [How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits](https://mlsanity.com/paper/2605.08348/) (2026)
- [Make Mechanistic Interpretability Auditable: A Call to Develop Guidelines via Continuous Collaborative Reviewing](https://mlsanity.com/paper/2606.00033/) (2026)
- [Circuit Claims Depend on What Is Extracted and How It Is Compared](https://mlsanity.com/paper/2607.18921/) (2026)
- [Identifying Sub-networks in Neural Networks via Functionally Similar Representations](https://mlsanity.com/paper/2410.16484/) (2024)
- [Direct Preference Optimization for Adaptive Concept-based Explanations](https://mlsanity.com/paper/2505.15626/) (2025)
- [Concept Concentration for Faithful Representation Intervention](https://mlsanity.com/paper/2505.18672/) (2025)
- [Circuit Distillation](https://mlsanity.com/paper/2509.25002/) (2025)
- [Comparing Explanations is Not Enough, Explain the Change: New Standards are Needed to Explain Behavioral Shifts in Large Language Models](https://mlsanity.com/paper/2602.02304/) (2026)
- [Mechanistically Interpreting Compression in Vision-Language Models](https://mlsanity.com/paper/2603.25035/) (2026)
- [How Modular Is a Frontier Mixture-of-Experts? A Pre-registered Causal Test in Which Apparent Expert Modularity Mostly Dissolves](https://mlsanity.com/paper/2606.25092/) (2026)
- [Position: Reasoning is a Learnable Rule-Based Process](https://mlsanity.com/paper/2608.12325/) (2026)
- [RACE: Scalable Statistical Estimation of Functional Consistency in LLM Neurons](https://mlsanity.com/paper/2608.24758/) (2026)
- [Rethinking Circuit Evaluation: Do Circuits Explain Model Errors?](https://mlsanity.com/paper/2609.35686/) (2026)
- [Are We Recovering Mechanisms? Objective-Level Recovery Gaps in Mechanistic Interpretability](https://mlsanity.com/paper/2610.02098/) (2026)

## Abstract (arXiv)

Large language models (LLMs) demonstrate surprising capabilities, but we do not understand how they are implemented. One hypothesis suggests that these capabilities are primarily executed by small subnetworks within the LLM, known as circuits. But how can we evaluate this hypothesis? In this paper, we formalize a set of criteria that a circuit is hypothesized to meet and develop a suite of hypothesis tests to evaluate how well circuits satisfy them. The criteria focus on the extent to which the LLM's behavior is preserved, the degree of localization of this behavior, and whether the circuit is minimal. We apply these tests to six circuits described in the research literature. We find that synthetic circuits -- circuits that are hard-coded in the model -- align with the idealized properties. Circuits discovered in Transformer models satisfy the criteria to varying degrees. To facilitate future empirical studies of circuits, we created the textitcircuitry package, a wrapper around the textitTransformerLens library, which abstracts away lower-level manipulations of hooks and activations. The software is available at urlhttps://github.com/blei-lab/circuitry.