# Many Circuits, One Mechanism: Input Variation and Evaluation Granularity in Circuit Discovery

arXiv: https://arxiv.org/abs/2606.06267 · HTML: https://mlsanity.com/paper/2606.06267/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability.
- Builds on 25 of its 111 references that are in the map.
- First posted on arXiv 4 Jun 2026.

## Placed in

- Mechanistic interpretability / [Circuits and mechanistic analysis](https://mlsanity.com/mechinterp/circuits-and-mechanistic-analysis/)

## Builds on (most cited first, 25 of 25)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [How to use and interpret activation patching](https://mlsanity.com/paper/2404.15255/) (2024)
- [Localizing Model Behavior with Path Patching](https://mlsanity.com/paper/2304.05969/) (2023)
- [The Hydra Effect: Emergent Self-repair in Language Model Computations](https://mlsanity.com/paper/2307.15771/) (2023)
- [Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla](https://mlsanity.com/paper/2307.09458/) (2023)
- [A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models](https://mlsanity.com/paper/2407.02646/) (2024)
- [AtP*: An efficient and scalable method for localizing LLM behaviour to components](https://mlsanity.com/paper/2403.00745/) (2024)
- [An Interpretability Illusion for BERT](https://mlsanity.com/paper/2104.07143/) (2021)
- [Missed Causes and Ambiguous Effects: Counterfactuals Pose Challenges for Interpreting Neural Networks](https://mlsanity.com/paper/2407.04690/) (2024)
- [Frequency-based Distortions in Contextualized Word Embeddings](https://mlsanity.com/paper/2104.08465/) (2021)
- [Sparse Autoencoders Enable Scalable and Reliable Circuit Identification in Language Models](https://mlsanity.com/paper/2405.12522/) (2024)
- [Mechanistic Interpretability as Statistical Estimation: A Variance Analysis](https://mlsanity.com/paper/2510.00845/) (2025)
- [Sheaf Discovery with Joint Computation Graph Pruning and Flexible Granularity](https://mlsanity.com/paper/2407.03779/) (2024)
- [Repetitions are not all alike: distinct mechanisms sustain repetition in language models](https://mlsanity.com/paper/2504.01100/) (2025)
- [Adaptive Circuit Behavior and Generalization in Mechanistic Interpretability](https://mlsanity.com/paper/2411.16105/) (2024)
- [Adversarial Circuit Evaluation](https://mlsanity.com/paper/2407.15166/) (2024)
- [The Fair Language Model Paradox](https://mlsanity.com/paper/2410.11985/) (2024)
- [Hierarchical Latent Structures in Data Generation Process Unify Mechanistic Phenomena across Scale](https://mlsanity.com/paper/2603.06592/) (2026)
- [Distributed Specialization: Rare-Token Neurons in Large Language Models](https://mlsanity.com/paper/2509.21163/) (2025)
- [Finding Interpretable Prompt-Specific Circuits in Language Models](https://mlsanity.com/paper/2602.13483/) (2026)

## Built on by (most cited first, 3 of 3)

- [Explanation Multiplicity: Circuit-Level Interpretability Evidence Does Not Survive Defensible Analytic Variation](https://mlsanity.com/paper/2608.13754/) (2026)
- [Circuit Condensation: Post-Training that Concentrates a Behavior's Causal Circuit](https://mlsanity.com/paper/2608.27254/) (2026)
- [SFT Leaves a Reproducible Signature in Activation Geometry: Calibrating Post-Training Interpretability Against Run-to-Run Noise](https://mlsanity.com/paper/2610.17093/) (2026)

## Abstract (arXiv)

Circuit discovery methods identify subgraphs that explain model behaviors, and structural differences between discovered circuits are commonly interpreted as evidence of distinct mechanisms. We test this assumption by varying input-token frequency while holding the task fixed. The discovered circuits appear specialized by frequency when compared structurally, but functional and representational analyses show no reliable evidence of corresponding differences. We term this mismatch phantom specialization. Using the Literal Sequence Copying task across four frequency bands plus a frequency-weighted control, we extract 75 circuits from five Pythia models (70M-1.4B). We find that structurally distinct circuits implement the same computation: band-specific edges transfer broadly across bands, a core shared across most bands recovers at least 99% of circuit performance in models above 70M, and causal interchange interventions confirm that internal representations are interchangeable across frequency bands. A smaller subject-verb agreement replication shows the same pattern: circuits differ structurally but transfer broadly across bands, and the core shared by most bands recovers nearly all circuit accuracy. Repeated extractions within the same band further suggest that discovery algorithms sample from an equivalence class of valid subgraphs rather than recovering a unique mechanism. Standard evaluation practice obscures this pattern: source-level evaluation inflates apparent faithfulness, while edge-level evaluation reveals the many-to-one mapping from structure to function. We do not test specialization at the level of token positions or features inside a component. Our results show that structural differences between circuits are not sufficient evidence for distinct mechanisms, and that exposing this requires edge-level evaluation and cross-condition transfer tests.