# The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching

arXiv: https://arxiv.org/abs/2606.27510 · HTML: https://mlsanity.com/paper/2606.27510/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability.
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 25 Jun 2026.

## Placed in

- Mechanistic interpretability / [Circuits and mechanistic analysis](https://mlsanity.com/mechinterp/circuits-and-mechanistic-analysis/)

## Builds on (most cited first, 24 of 24)

- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021)
- [Direct and Indirect Effects](https://mlsanity.com/paper/1301.2300/) (2013)
- [How to use and interpret activation patching](https://mlsanity.com/paper/2404.15255/) (2024)
- [Attribution Patching Outperforms Automated Circuit Discovery](https://mlsanity.com/paper/2310.10348/) (2023)
- [How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model](https://mlsanity.com/paper/2305.00586/) (2023)
- [Localizing Model Behavior with Path Patching](https://mlsanity.com/paper/2304.05969/) (2023)
- [Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms](https://mlsanity.com/paper/2403.17806/) (2024)
- [Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models](https://mlsanity.com/paper/2106.06087/) (2021)
- [SAEs Are Good for Steering -- If You Select the Right Features](https://mlsanity.com/paper/2505.20063/) (2025)
- [A Multimodal Automated Interpretability Agent](https://mlsanity.com/paper/2404.14394/) (2024)
- [MIB: A Mechanistic Interpretability Benchmark](https://mlsanity.com/paper/2504.13151/) (2025)
- [Hypothesis Testing the Circuit Hypothesis in LLMs](https://mlsanity.com/paper/2410.13032/) (2024)
- [InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniques](https://mlsanity.com/paper/2407.14494/) (2024)
- [The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysis](https://mlsanity.com/paper/2408.01416/) (2024)
- [Missed Causes and Ambiguous Effects: Counterfactuals Pose Challenges for Interpreting Neural Networks](https://mlsanity.com/paper/2407.04690/) (2024)
- [Because we have LLMs, we Can and Should Pursue Agentic Interpretability](https://mlsanity.com/paper/2506.12152/) (2025)
- [Mechanistic Interpretability as Statistical Estimation: A Variance Analysis](https://mlsanity.com/paper/2510.00845/) (2025)
- [The Shape of Beliefs: Geometry, Dynamics, and Interventions along Representation Manifolds of Language Models' Posteriors](https://mlsanity.com/paper/2602.02315/) (2026)
- [Insights into the "cross-world" independence assumption of causal mediation analysis](https://mlsanity.com/paper/2003.10341/) (2020)
- [Pitfalls in Evaluating Interpretability Agents](https://mlsanity.com/paper/2603.20101/) (2026)

## Built on by (most cited first, 11 of 11)

- [Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuits](https://mlsanity.com/paper/2607.01940/) (2026)
- [Can Neural Networks Learn by Experimenting on Themselves? Self-Interventional Learning from Functional Consequences to Predictive Self-Knowledge](https://mlsanity.com/paper/2608.14894/) (2026)
- [A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation](https://mlsanity.com/paper/2608.03620/) (2026)
- [Cross-Layer Interaction under Weight-Space Ablation: A Closed-Form Attention Jacobian Bound and a Test on a Real Pretrained Model](https://mlsanity.com/paper/2608.03629/) (2026)
- [Reading the Gate, Not the Interference: Output-Side Interference Measurement Does Not Track Merge Collapse](https://mlsanity.com/paper/2608.11797/) (2026)
- [ObserverBench: Testing Mechanistic Estimates for Intervention and Control](https://mlsanity.com/paper/2609.03026/) (2026)
- [Activation-Space Order-Swap Geometry: A Site-Asymmetry Audit](https://mlsanity.com/paper/2608.25315/) (2026)
- [Exact Feedback Is Not Control: Evaluating Text-based Closed-Loop Revision in LLMs](https://mlsanity.com/paper/2609.28150/) (2026)
- [Right Answer, Wrong Mechanism: Detecting Pernicious Divergence in Causal Interventions](https://mlsanity.com/paper/2609.39243/) (2026)
- [Every Ablation Is a Dose: Counterweights and the Semblance of Self-Repair](https://mlsanity.com/paper/2610.02173/) (2026)
- [Slaying the Hydra: Interaction-Aware Circuit Discovery in Language Models](https://mlsanity.com/paper/2610.04017/) (2026)

## Abstract (arXiv)

Activation patching is the primary tool in mechanistic interpretability. It attributes causal responsibility for a model behavior to each of its individual components by estimating its natural indirect effect (NIE). Re-deriving the activation patching estimand from causal mediation analysis, we find that the NIE does not solely capture the causal effect through the specific component. It also contains interaction effects (INT) that measure how much the component's causal effect itself depends on the state of other components in the model. A natural response may be to try to eliminate INT by adjusting the estimator or unit of analysis, but each of these potential remedies has predictable failure modes. We demonstrate these failure modes in the GPT-2 IOI circuit; components whose causal importance is conditional on the state of other components are either invisible or artificially inflated, and INT variance explains the previously documented instability of faithfulness scores. We prove that INT scales with the distance between clean and patched component activations, is negligible when the model is locally affine, and decomposes combinatorially into pairwise and higher-order group interactions. Despite its inevitability, INT is not a nuisance to be eliminated, but rather a diagnostic for interpretability studies. Its individual and group-level magnitude and sign signal when causal conclusions are prompt-dependent, and when greedy NIE-based component ranking will miss mechanisms only discoverable through combinatorial search.