# Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuits

arXiv: https://arxiv.org/abs/2607.01940 · HTML: https://mlsanity.com/paper/2607.01940/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability.
- Builds on 33 of its 46 references that are in the map.
- First posted on arXiv 2 Jul 2026.

## Placed in

- Mechanistic interpretability / [Circuits and mechanistic analysis](https://mlsanity.com/mechinterp/circuits-and-mechanistic-analysis/)

## Builds on (most cited first, 30 of 33)

- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [A Simple and Effective Pruning Approach for Large Language Models](https://mlsanity.com/paper/2306.11695/) (2023)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Importance Estimation for Neural Network Pruning](https://mlsanity.com/paper/1906.10771/) (2019)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [Attribution Patching Outperforms Automated Circuit Discovery](https://mlsanity.com/paper/2310.10348/) (2023)
- [How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model](https://mlsanity.com/paper/2305.00586/) (2023)
- [A Fast Post-Training Pruning Framework for Transformers](https://mlsanity.com/paper/2204.09656/) (2022)
- [The Hydra Effect: Emergent Self-repair in Language Model Computations](https://mlsanity.com/paper/2307.15771/) (2023)
- [Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms](https://mlsanity.com/paper/2403.17806/) (2024)
- [AtP*: An efficient and scalable method for localizing LLM behaviour to components](https://mlsanity.com/paper/2403.00745/) (2024)
- [Neuron Shapley: Discovering the Responsible Neurons](https://mlsanity.com/paper/2002.09815/) (2020)
- [Information Flow Routes: Automatically Interpreting Language Models at Scale](https://mlsanity.com/paper/2403.00824/) (2024)
- [Finding Transformer Circuits with Edge Pruning](https://mlsanity.com/paper/2406.16778/) (2024)
- [Explorations of Self-Repair in Language Models](https://mlsanity.com/paper/2402.15390/) (2024)
- [Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition](https://mlsanity.com/paper/2407.00886/) (2024)
- [Transformer Circuit Faithfulness Metrics are not Robust](https://mlsanity.com/paper/2407.08734/) (2024)
- [The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching](https://mlsanity.com/paper/2606.27510/) (2026)
- [Rethinking Circuit Completeness in Language Models: AND, OR, and ADDER Gates](https://mlsanity.com/paper/2505.10039/) (2025)
- [A Brain-like Synergistic Core in LLMs Drives Behaviour and Learning](https://mlsanity.com/paper/2601.06851/) (2026)
- [Correcting Gradient-Based Circuit Localization via Interaction-Aware Backpropagation](https://mlsanity.com/paper/2505.17630/) (2025)
- [Measuring Affinity between Attention-Head Weight Subspaces via the Projection Kernel](https://mlsanity.com/paper/2601.10266/) (2026)

## Built on by (most cited first, 9 of 9)

- [CoCurve: Cross-Module Co-Pruning Curvature for Structured LLM Pruning](https://mlsanity.com/paper/2607.17568/) (2026)
- [A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation](https://mlsanity.com/paper/2608.03620/) (2026)
- [Cross-Layer Interaction under Weight-Space Ablation: A Closed-Form Attention Jacobian Bound and a Test on a Real Pretrained Model](https://mlsanity.com/paper/2608.03629/) (2026)
- [The Communication Map of a Transformer](https://mlsanity.com/paper/2608.22007/) (2026)
- [State of Thought Enables Endogenous Reasoning](https://mlsanity.com/paper/2609.16055/) (2026)
- [Representational Simplicity and Circuit Size Dissociate in a Threshold-Dependent Way: A Controlled Test via Adversarial Training](https://mlsanity.com/paper/2609.35890/) (2026)
- [Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers](https://mlsanity.com/paper/2609.39259/) (2026)
- [Every Ablation Is a Dose: Counterweights and the Semblance of Self-Repair](https://mlsanity.com/paper/2610.02173/) (2026)
- [Slaying the Hydra: Interaction-Aware Circuit Discovery in Language Models](https://mlsanity.com/paper/2610.04017/) (2026)

## Abstract (arXiv)

Mechanistic interpretability seeks to explain transformer behavior through circuits: sets of internal components that causally support a behavior. However, self-repair creates a blind spot: ablating a primary component can activate a dormant backup, so a circuit that explains behavior in the intact model can become incomplete under the intervention used to test it. We formulate this gap as conditional circuit completion: given a primary set, identify components that become causally important after its removal. We introduce conditional co-ablation (CoAx), which ranks candidates by growth in ablation effect after primary-set removal. We show that a perfectly dormant backup can be indistinguishable from an irrelevant component to per-unit intact-state scores, whereas its conditional effect change exactly aggregates all interaction orders linking it to the removed set. On GPT-2-small's Indirect Object Identification (IOI) circuit, CoAx recovers the documented backup heads at 0.941 ROC-AUC, versus 0.815 for the strongest intact-state attribution baseline and 0.758 for the matched conditional-energy control. Recovery drops to 0.40 +/- 0.13 AUC for alternative component sets matched in behavioral effect, output displacement, and depth, showing that recovery is specific to the removed circuit. Beyond recovery, the CoAx-selected heads are causally load-bearing: freezing them after primary removal sharply reduces the IOI margin, while adding them to the incomplete circuit reduces incompleteness from 0.75 to 0.21. More broadly, conditional growth aligns with intervention-derived repair in 11/12 held-out instances across 4 mechanism clusters, and CoAx completions outperform matched random completions on all 8 non-GPT-2 models spanning 6 architecture families. Together, causal explanations of self-repairing transformers must account for backup circuitry when primary components fail.