mlsanityData to 14 Oct 2026

How to use and interpret activation patching

arXiv 2404.15255 ↗First posted 23 Apr 2024cs.LGcited by 214

Not on the map. Why?

Cited by 77 in Mechanistic interpretability, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Activation patching is a popular mechanistic interpretability technique, but has many subtleties regarding how it is applied and how one may interpret the results. We provide a summary of advice and best practices, based on our experience using this technique in practice. We include an overview of the different ways to apply activation patching and a discussion on how to interpret the results. We focus on what evidence patching experiments provide about circuits, and on the choice of metric and associated pitfalls.

Key facts. Cited by 214 papers in the map (data through 14 Oct 2026). Builds on 24 of its 29 references that are in the map. First posted on arXiv 23 Apr 2024.

Builds on

24 of 29 references.

References · 24

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  3. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  4. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  5. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  6. 2021Causal Abstractions of Neural Networkscited by 422
  7. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  8. 2023In-Context Learning Creates Task Vectorscited by 312
  9. 2023Function Vectors in Large Language Modelscited by 308
  10. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 243
  11. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 202
  12. 2023Localizing Model Behavior with Path Patchingcited by 195
12 more
  1. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 191
  2. 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 156
  3. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 149
  4. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 131
  5. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 110
  6. 2023Circuit Component Reuse Across Tasks in Transformer Language Modelscited by 108
  7. 2023How do Language Models Bind Entities in Context?cited by 108
  8. 2023A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysiscited by 89
  9. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 87
  10. 2023Copy Suppression: Comprehensively Understanding an Attention Headcited by 76
  11. 2023Rigorously Assessing Natural Language Explanations of Neuronscited by 45
  12. 2019Discovering the Compositional Structure of Vector Representations with Role Learning Networkscited by 34

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 23; 2025: 63; 2026: 128.

214 papers.

Sort:

By area:

  1. 2024Transcoders Find Interpretable LLM Feature Circuitscited by 200
  2. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 118
  3. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 115
  4. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
  5. 2024Attention Heads of Large Language Models: A Surveycited by 67
  6. 2025Weight-sparse transformers have interpretable circuitscited by 44
  7. 2024Interpreting Attention Layer Outputs with Sparse Autoencoderscited by 43
  8. 2025Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activationscited by 41
  9. 2025(How) Do Language Models Track State?cited by 34
  10. 2024Controlling Large Language Model Agents with Entropic Activation Steeringcited by 29
  11. 2024Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inferencecited by 29
  12. 2025Mechanistic Interpretability of Emotion Inference in Large Language Modelscited by 27
38 more
  1. 2026How do LLMs Compute Verbal Confidencecited by 26
  2. 2024Optimal ablation for interpretabilitycited by 25
  3. 2024Missed Causes and Ambiguous Effects: Counterfactuals Pose Challenges for Interpreting Neural Networkscited by 23
  4. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 21
  5. 2024Transformers Struggle to Learn to Searchcited by 19
  6. 2025How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencodingcited by 19
  7. 2024Compact Proofs of Model Performance via Mechanistic Interpretabilitycited by 18
  8. 2024Exploring and steering the moral compass of Large Language Modelscited by 16
  9. 2025Consistency Training Helps Stop Sycophancy and Jailbreakscited by 16
  10. 2024Sparse Autoencoders Reveal Temporal Difference Learning in Large Language Modelscited by 15
  11. 2024A Implies B: Circuit Analysis in LLMs for Propositional Logical Reasoningcited by 15
  12. 2025Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Racecited by 15
  13. 2025Beyond Transcription: Mechanistic Interpretability in ASRcited by 13
  14. 2025How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decodingcited by 13
  15. 2025Analogical Reasoning Inside Large Language Models: Concept Vectors and the Limits of Abstractioncited by 12
  16. 2024Racing Thoughts: Explaining Contextualization Errors in Large Language Modelscited by 11
  17. 2025Transformer Dynamics: A neuroscientific approach to interpretability of large language modelscited by 11
  18. 2025Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?cited by 11
  19. 2026The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patchingcited by 11
  20. 2025Rethinking Circuit Completeness in Language Models: AND, OR, and ADDER Gatescited by 10
  21. 2024Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspectivecited by 9
  22. 2025Interpreting Attention Heads for Image-to-Text Information Flow in Large Vision-Language Modelscited by 9
  23. 2025Elucidating Mechanisms of Demographic Bias in LLMs for Healthcarecited by 8
  24. 2026Causality is Key for Interpretability Claims to Generalisecited by 8
  25. 2026Steered LLM Activations are Non-Surjectivecited by 8
  26. 2025How do Large Language Models Understand Relevance? A Mechanistic Interpretability Perspectivecited by 7
  27. 2025Are We Paying Attention to Her? Investigating Gender Disambiguation and Attention in Machine Translationcited by 7
  28. 2025Transformers Don't Need LayerNorm at Inference Time: Scaling LayerNorm Removal to GPT-2 XL and the Implications for Mechanistic Interpretabilitycited by 7
  29. 2025CLUE: Conflict-guided Localization for LLM Unlearning Frameworkcited by 7
  30. 2025Addressing divergent representations from causal interventions on neural networkscited by 7
  31. 2025Beyond Components: Singular Vector-Based Interpretability of Transformer Circuitscited by 7
  32. 2025On Geometric Understanding and Learned Priors in Feed-forward 3D Reconstruction Modelscited by 7
  33. 2025Emergent Symbol-like Number Variables in Artificial Neural Networkscited by 6
  34. 2025Designing Role Vectors to Improve LLM Inference Behaviourcited by 6
  35. 2025Why Safeguarded Ships Run Aground? Aligned Large Language Models' Safety Mechanisms Tend to Be Anchored in The Template Regioncited by 6
  36. 2025Circuit Stability Characterizes Language Model Generalizationcited by 6
  37. 2025Rethinking Explainability in the Era of Multimodal AIcited by 6
  38. 2025Large Language Models as Psychological Simulators: A Methodological Guidecited by 6

50 most cited of 214 All 214 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Patch-Effect Graph Kernels for LLM Interpretability
  2. 2023Attribution Patching Outperforms Automated Circuit Discovery
  3. 2023Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching
  4. 2025Steering Awareness: Detecting Activation Steering from Within
  5. 2026Attention Gathers, MLPs Compose: A Causal Analysis of an Action-Outcome Circuit in VideoViT
  6. 2026From Attribution to Action: A Human-Centered Application of Activation Steering
  7. 2024Transformer Circuit Faithfulness Metrics are not Robust
  8. 2025APP: Accelerated Path Patching with Task-Specific Pruning