mlsanityData to 7 Oct 2026

Localizing Model Behavior with Path Patching

arXiv 2304.05969 ↗First posted 12 Apr 2023cs.LGcited by 182

Not on the map. Why?

Abstract (arXiv)

Localizing behaviors of neural networks to a subset of the network's components or a subset of interactions between components is a natural first step towards analyzing network mechanisms and possible failure modes. Existing work is often qualitative and ad-hoc, and there is no consensus on the appropriate way to evaluate localization claims. We introduce path patching, a technique for expressing and quantitatively testing a natural class of hypotheses expressing that behaviors are localized to a set of paths. We refine an explanation of induction heads, characterize a behavior of GPT-2, and open source a framework for efficiently running similar experiments.

Key facts. Cited by 182 papers in the map (data through 7 Oct 2026). Builds on 16 of its 29 references that are in the map. First posted on arXiv 12 Apr 2023.

Builds on

16 of 29 references.

References · 16

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  2. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  3. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  4. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  5. 2021Causal Abstractions of Neural Networkscited by 401
  6. 2013Direct and Indirect Effectscited by 276
  7. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 197
  8. 2023A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operationscited by 141
  9. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 127
  10. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  11. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  12. 2021Scale Efficiently: Insights from Pre-training and Fine-tuning Transformerscited by 105
4 more
  1. 2023Tracr: Compiled Transformers as a Laboratory for Interpretabilitycited by 99
  2. 2021An Interpretability Illusion for BERTcited by 94
  3. 2020Shortformer: Better Language Modeling using Shorter Inputscited by 76
  4. 2019Approximate Causal Abstractioncited by 62

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 20; 2024: 50; 2025: 37; 2026: 75.

182 papers.

Sort:

By area:

  1. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  2. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  3. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  4. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  5. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  6. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  7. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 199
  8. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  9. 2024How to use and interpret activation patchingcited by 191
  10. 2024Transcoders Find Interpretable LLM Feature Circuitscited by 186
  11. 2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 140
  12. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
38 more
  1. 2023Circuit Component Reuse Across Tasks in Transformer Language Modelscited by 105
  2. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 100
  3. 2024Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Modelscited by 87
  4. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  5. 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
  6. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 77
  7. 2023Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patchingcited by 75
  8. 2023Copy Suppression: Comprehensively Understanding an Attention Headcited by 73
  9. 2023Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIPcited by 62
  10. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  11. 2024LLM Circuit Analyses Are Consistent Across Training and Scalecited by 60
  12. 2024Finding Transformer Circuits with Edge Pruningcited by 58
  13. 2024pyvene: A Library for Understanding and Improving PyTorch Models via Interventionscited by 53
  14. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  15. 2024CausalGym: Benchmarking causal interpretability methods on linguistic taskscited by 45
  16. 2024Interpreting Attention Layer Outputs with Sparse Autoencoderscited by 43
  17. 2024Interpreting and Improving Large Language Models in Arithmetic Calculationcited by 43
  18. 2024Talking Heads: Understanding Inter-layer Communication in Transformer Language Modelscited by 39
  19. 2024Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficientcited by 39
  20. 2023Neuron-Level Knowledge Attribution in Large Language Modelscited by 37
  21. 2024Personality Alignment of Large Language Modelscited by 36
  22. 2024Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localizationcited by 34
  23. 2023Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarkscited by 29
  24. 2023Circuit Breaking: Removing Model Behaviors with Targeted Ablationcited by 27
  25. 2024The Same But Different: Structural Similarities and Differences in Multilingual Language Modelingcited by 27
  26. 2024Transformers on Markov Data: Constant Depth Sufficescited by 26
  27. 2023Interpretability Illusions in the Generalization of Simplified Modelscited by 24
  28. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
  29. 2024Efficient Automated Circuit Discovery in Transformers using Contextual Decompositioncited by 23
  30. 2024Locking Down the Finetuned LLMs Safetycited by 22
  31. 2024Controllable Context Sensitivity and the Knob Behind Itcited by 22
  32. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  33. 2024Missed Causes and Ambiguous Effects: Counterfactuals Pose Challenges for Interpreting Neural Networkscited by 21
  34. 2024Optimal ablation for interpretabilitycited by 21
  35. 2024Decomposing and Editing Predictions by Modeling Model Computationcited by 19
  36. 2025How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencodingcited by 19
  37. 2024Transformer Circuit Faithfulness Metrics are not Robustcited by 18
  38. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 18

50 most cited of 182 All 182 (JSON)