mlsanityData to 14 Oct 2026

The Hydra Effect: Emergent Self-repair in Language Model Computations

arXiv 2307.15771 ↗First posted 28 Jul 2023cs.LG cs.AI cs.CLcited by 156

Not on the map. Why?

Cited by 44 in Mechanistic interpretability, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We investigate the internal structure of language model computations using causal analysis and demonstrate two motifs: (1) a form of adaptive computation where ablations of one attention layer of a language model cause another layer to compensate (which we term the Hydra effect) and (2) a counterbalancing function of late MLP layers that act to downregulate the maximum-likelihood token. Our ablation studies demonstrate that language model layers are typically relatively loosely coupled (ablations to one layer only affect a small number of downstream layers). Surprisingly, these effects occur even in language models trained without any form of dropout. We analyse these effects in the context of factual recall and consider their implications for circuit-level attribution in language models.

Key facts. Cited by 156 papers in the map (data through 14 Oct 2026). Builds on 31 of its 54 references that are in the map. First posted on arXiv 28 Jul 2023.

Builds on

31 of 54 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Layer Normalizationcited by 5,669
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  4. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  6. 2019Root Mean Square Layer Normalizationcited by 1,480
  7. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  8. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  9. 2022Toy Models of Superpositioncited by 971
  10. 2022In-context Learning and Induction Headscited by 964
  11. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  12. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
19 more
  1. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  2. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 629
  3. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  4. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  5. 2021Causal Abstractions of Neural Networkscited by 422
  6. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  7. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
  8. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
  9. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 191
  10. 2023A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operationscited by 144
  11. 2022Analyzing Transformers in Embedding Spacecited by 132
  12. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 131
  13. 2021Acquisition of Chess Knowledge in AlphaZerocited by 123
  14. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 110
  15. 2018On the importance of single directions for generalizationcited by 99
  16. 2022Formal Algorithms for Transformerscited by 72
  17. 2020Towards falsifiable interpretability researchcited by 53
  18. 2022LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Modelscited by 28
  19. 2022Causal Abstraction with Soft Interventionscited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 13; 2024: 42; 2025: 26; 2026: 75.

156 papers.

Sort:

By area:

  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  2. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  3. 2023Explainability for Large Language Models: A Surveycited by 333
  4. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  5. 2024How to use and interpret activation patchingcited by 214
  6. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 169
  7. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 146
  8. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 111
  9. 2024Universal Neurons in GPT2 Language Modelscited by 102
  10. 2024Information Flow Routes: Automatically Interpreting Language Models at Scalecited by 95
  11. 2024Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Controlcited by 86
  12. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
38 more
  1. 2024Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queriescited by 78
  2. 2025Do Multilingual LLMs Think In English?cited by 78
  3. 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
  4. 2023Copy Suppression: Comprehensively Understanding an Attention Headcited by 76
  5. 2024LLM Circuit Analyses Are Consistent Across Training and Scalecited by 63
  6. 2024Finding Transformer Circuits with Edge Pruningcited by 60
  7. 2024LoFiT: Localized Fine-tuning on LLM Representationscited by 57
  8. 2024How to think step-by-step: A mechanistic understanding of chain-of-thought reasoningcited by 53
  9. 2024Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neuronscited by 53
  10. 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 53
  11. 2024Everybody Prune Now: Structured Pruning of LLMs with only Forward Passescited by 43
  12. 2024Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMscited by 42
  13. 2024Dissecting Language Models: Machine Unlearning via Selective Pruningcited by 42
  14. 2024Explorations of Self-Repair in Language Modelscited by 39
  15. 2024Interpreting the Second-Order Effects of Neurons in CLIPcited by 36
  16. 2024Gradient Routing: Masking Gradients to Localize Computation in Neural Networkscited by 34
  17. 2024Interpreting Key Mechanisms of Factual Recall in Transformer-Based Language Modelscited by 31
  18. 2024Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPTcited by 29
  19. 2024Optimal ablation for interpretabilitycited by 25
  20. 2025Beyond Linear Steering: Unified Multi-Attribute Control for Language Modelscited by 25
  21. 2023Grokking Group Multiplication with Cosetscited by 24
  22. 2024Automatically Identifying Local and Global Circuits with Linear Computation Graphscited by 24
  23. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
  24. 2024Missed Causes and Ambiguous Effects: Counterfactuals Pose Challenges for Interpreting Neural Networkscited by 23
  25. 2023Attention Lens: A Tool for Mechanistically Interpreting the Attention Head Information Retrieval Mechanismcited by 22
  26. 2024Language Models Encode the Value of Numbers Linearlycited by 20
  27. 2025Beyond I'm Sorry, I Can't: Dissecting Large Language Model Refusalcited by 19
  28. 2023Look Before You Leap: A Universal Emergent Decomposition of Retrieval Tasks in Language Modelscited by 18
  29. 2024Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neuronscited by 16
  30. 2025Stochastic Parameter Decompositioncited by 16
  31. 2024Monotonic Representation of Numeric Properties in Language Modelscited by 15
  32. 2024Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformerscited by 15
  33. 2024A Implies B: Circuit Analysis in LLMs for Propositional Logical Reasoningcited by 15
  34. 2025Implicit Reasoning in Transformers is Reasoning through Shortcutscited by 14
  35. 2024Detecting and Understanding Vulnerabilities in Language Models via Mechanistic Interpretabilitycited by 13
  36. 2025Analyze Feature Flow to Enhance Interpretation and Steering in Language Modelscited by 13
  37. 2025The Dead Salmons of AI Interpretabilitycited by 12
  38. 2025Towards mechanistic understanding in a data-driven weather model: internal activations reveal interpretable physical featurescited by 12

50 most cited of 156 All 156 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024A Theoretical Understanding of Self-Correction through In-context Alignment
  2. 2026MechaTerp-TRACE: A Novel Approach for Component Ablation Analysis in Language Models
  3. 2026Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay
  4. 2026Balancing Memory Pathways: Analyzing and Improving Memory Utilization in Hybrid LMs
  5. 2026It's Not RoPE that Creates Sinks: The Role of Self-Concentration and Value-Non-Mixing in Attention
  6. 2025Forgetting as a Feature: Cognitive Alignment of Large Language Models
  7. 2026Sensitivity, Causality, and Repair Dissociate: A Layer-Wise Analysis of Perturbation Robustness and Its Scaling
  8. 2025Disentangling Recall and Reasoning in Transformer Models through Layer-wise Attention and Activation Analysis