# Explorations of Self-Repair in Language Models

arXiv: https://arxiv.org/abs/2402.15390 · HTML: https://mlsanity.com/paper/2402.15390/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 23 Feb 2024.

## Builds on (most cited first, 14 of 14)

- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023)
- [Understanding the Role of Individual Units in a Deep Neural Network](https://mlsanity.com/paper/2009.05041/) (2020)
- [Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias](https://mlsanity.com/paper/2004.12265/) (2020)
- [The Hydra Effect: Emergent Self-repair in Language Model Computations](https://mlsanity.com/paper/2307.15771/) (2023)
- [Residual Connections Encourage Iterative Inference](https://mlsanity.com/paper/1710.04773/) (2017)
- [Highway and Residual Networks learn Unrolled Iterative Estimation](https://mlsanity.com/paper/1612.07771/) (2016)

## Built on by (most cited first, 30 of 39)

- [The Remarkable Robustness of LLMs: Stages of Inference?](https://mlsanity.com/paper/2406.19384/) (2024)
- [A Primer on the Inner Workings of Transformer-based Language Models](https://mlsanity.com/paper/2405.00208/) (2024)
- [The Geometry of Concepts: Sparse Autoencoder Feature Structure](https://mlsanity.com/paper/2410.19750/) (2024)
- [Optimal ablation for interpretability](https://mlsanity.com/paper/2409.09951/) (2024)
- [Monotonic Representation of Numeric Properties in Language Models](https://mlsanity.com/paper/2403.10381/) (2024)
- [Looking Beyond The Top-1: Transformers Determine Top Tokens In Order](https://mlsanity.com/paper/2410.20210/) (2024)
- [Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?](https://mlsanity.com/paper/2510.06036/) (2025)
- [Is One Layer Enough? Understanding Inference Dynamics in Tabular Foundation Models](https://mlsanity.com/paper/2605.06510/) (2026)
- [Interpreting Attention Heads for Image-to-Text Information Flow in Large Vision-Language Models](https://mlsanity.com/paper/2509.17588/) (2025)
- [Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuits](https://mlsanity.com/paper/2607.01940/) (2026)
- [From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Models](https://mlsanity.com/paper/2407.09502/) (2024)
- [Correcting Gradient-Based Circuit Localization via Interaction-Aware Backpropagation](https://mlsanity.com/paper/2505.17630/) (2025)
- [Sparse Attention Decomposition Applied to Circuit Tracing](https://mlsanity.com/paper/2410.00340/) (2024)
- [Neuroplasticity and Corruption in Model Mechanisms: A Case Study Of Indirect Object Identification](https://mlsanity.com/paper/2503.01896/) (2025)
- [What Iterated Self-Feeding Probes of Language Models Measure, and a test that separates the construction from the model](https://mlsanity.com/paper/2608.10986/) (2026)
- [LLM Layers Immediately Correct Each Other](https://mlsanity.com/paper/2609.07876/) (2026)
- [Decoding Vision Transformers: the Diffusion Steering Lens](https://mlsanity.com/paper/2504.13763/) (2025)
- [Hierarchical Latent Structures in Data Generation Process Unify Mechanistic Phenomena across Scale](https://mlsanity.com/paper/2603.06592/) (2026)
- [Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning](https://mlsanity.com/paper/2606.07560/) (2026)
- [Model Hemorrhage and the Robustness Limits of Large Language Models](https://mlsanity.com/paper/2503.23924/) (2025)
- [Understanding Gated Neurons in Transformers from Their Input-Output Functionality](https://mlsanity.com/paper/2505.17936/) (2025)
- [Reading the Gate, Not the Interference: Output-Side Interference Measurement Does Not Track Merge Collapse](https://mlsanity.com/paper/2608.11797/) (2026)
- [ObserverBench: Testing Mechanistic Estimates for Intervention and Control](https://mlsanity.com/paper/2609.03026/) (2026)
- [Through the Looking Glass: Directly Reading and Writing Transformers](https://mlsanity.com/paper/2609.10210/) (2026)
- [Time Course MechInterp: Analyzing the Evolution of Components and Knowledge in Large Language Models](https://mlsanity.com/paper/2506.03434/) (2025)
- [Beyond Transfer Accuracy: Mechanism-Guided Controlled Adaptation for Low-Resource Languages](https://mlsanity.com/paper/2601.08146/) (2026)
- [Capability-Guided Compression: Toward Interpretability-Aware Budget Allocation for Large Language Models](https://mlsanity.com/paper/2603.16440/) (2026)
- [In-Context Fixation: When Demonstrated Labels Override Semantics in Few-Shot Classification](https://mlsanity.com/paper/2605.08295/) (2026)
- [Can Graph Learning Learn Circuits?](https://mlsanity.com/paper/2608.08536/) (2026)
- [Do VLMs Share Safety Neurons Across Modalities?](https://mlsanity.com/paper/2608.30750/) (2026)

## Abstract (arXiv)

Prior interpretability research studying narrow distributions has preliminarily identified self-repair, a phenomena where if components in large language models are ablated, later components will change their behavior to compensate. Our work builds off this past literature, demonstrating that self-repair exists on a variety of models families and sizes when ablating individual attention heads on the full training distribution. We further show that on the full training distribution self-repair is imperfect, as the original direct effect of the head is not fully restored, and noisy, since the degree of self-repair varies significantly across different prompts (sometimes overcorrecting beyond the original effect). We highlight two different mechanisms that contribute to self-repair, including changes in the final LayerNorm scaling factor and sparse sets of neurons implementing Anti-Erasure. We additionally discuss the implications of these results for interpretability practitioners and close with a more speculative discussion on the mystery of why self-repair occurs in these models at all, highlighting evidence for the Iterative Inference hypothesis in language models, a framework that predicts self-repair.