mlsanityData to 7 Oct 2026

Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability

arXiv 2301.04709 ↗First posted 11 Jan 2023cs.AIcited by 194

Not on the map. Why?

Abstract (arXiv)

Causal abstraction provides a theoretical foundation for mechanistic interpretability, the field concerned with providing intelligible algorithms that are faithful simplifications of the known, but opaque low-level details of black box AI models. Our contributions are (1) generalizing the theory of causal abstraction from mechanism replacement (i.e., hard and soft interventions) to arbitrary mechanism transformation (i.e., functionals from old mechanisms to new mechanisms), (2) providing a flexible, yet precise formalization for the core concepts of polysemantic neurons, the linear representation hypothesis, modular features, and graded faithfulness, and (3) unifying a variety of mechanistic interpretability methods in the common language of causal abstraction, namely, activation and path patching, causal mediation analysis, causal scrubbing, causal tracing, circuit analysis, concept erasure, sparse autoencoders, differential binary masking, distributed alignment search, and steering.

Key facts. Cited by 194 papers in the map (data through 7 Oct 2026). Builds on 65 of its 136 references that are in the map. First posted on arXiv 11 Jan 2023.

Builds on

65 of 136 references.

References · 65

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  6. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  7. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  8. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  9. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  10. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  11. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  12. 2014Striving for Simplicity: The All Convolutional Netcited by 983
38 more
  1. 2022Mass-Editing Memory in a Transformercited by 976
  2. 2022In-context Learning and Induction Headscited by 930
  3. 2022Toy Models of Superpositioncited by 928
  4. 2019Are Sixteen Heads Really Better than One?cited by 803
  5. 2016The Mythos of Model Interpretabilitycited by 716
  6. 2019Attention is not not Explanationcited by 629
  7. 2019Designing and Interpreting Probes with Control Taskscited by 608
  8. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  9. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  10. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  11. 2021Causal Abstractions of Neural Networkscited by 401
  12. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 295
  13. 2018Consistent Individualized Feature Attribution for Tree Ensemblescited by 279
  14. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  15. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
  16. 2020Information-Theoretic Probing for Linguistic Structurecited by 204
  17. 2016Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layerscited by 197
  18. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 183
  19. 2021Implicit Representations of Meaning in Neural Language Modelscited by 179
  20. 2020A Theory of Usable Information Under Computational Constraintscited by 153
  21. 2018Dissecting Contextual Word Embeddings: Architecture and Representationcited by 146
  22. 2020Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?cited by 144
  23. 2016Foundations of Structural Causal Models with Cycles and Latent Variablescited by 134
  24. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  25. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 127
  26. 2020CausaLM: Causal Model Explanation Through Counterfactual Language Modelscited by 117
  27. 2017Causal Consistency of Structural Equation Modelscited by 114
  28. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  29. 2018Abstracting Causal Modelscited by 104
  30. 2018Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Informationcited by 97
  31. 2020Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Maskscited by 97
  32. 2021An Interpretability Illusion for BERTcited by 94
  33. 2019Neural Network Attributions: A Causal Perspectivecited by 77
  34. 2022Polysemanticity and Capacity in Neural Networkscited by 74
  35. 2020How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Maskingcited by 63
  36. 2021Low-Complexity Probing via Finding Subnetworkscited by 63
  37. 2019Approximate Causal Abstractioncited by 62
  38. 2021Conditional probing: measuring usable information beyond a baselinecited by 61

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 13; 2024: 22; 2025: 73; 2026: 86.

194 papers.

Sort:

By area:

  1. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  2. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  3. 2024ReFT: Representation Finetuning for Language Modelscited by 186
  4. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 145
  5. 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 145
  6. 2025Thought Anchors: Which LLM Reasoning Steps Matter?cited by 126
  7. 2023Interpretability at Scale: Identifying Causal Mechanisms in Alpacacited by 102
  8. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 100
  9. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  10. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 73
  11. 2023Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIPcited by 62
  12. 2025How Linguistics Learned to Stop Worrying and Love the Language Modelscited by 50
38 more
  1. 2025MIB: A Mechanistic Interpretability Benchmarkcited by 49
  2. 2024CausalGym: Benchmarking causal interpretability methods on linguistic taskscited by 45
  3. 2024Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representationscited by 45
  4. 2023Rigorously Assessing Natural Language Explanations of Neuronscited by 44
  5. 2025Enhancing Automated Interpretability with Output-Centric Feature Descriptionscited by 39
  6. 2025Language Models use Lookbacks to Track Beliefscited by 36
  7. 2024Evaluating Open-Source Sparse Autoencoders on Disentangling Factual Knowledge in GPT-2 Smallcited by 34
  8. 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?cited by 34
  9. 2024Mechanistic?cited by 25
  10. 2023A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepediacited by 24
  11. 2025Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Contextcited by 24
  12. 2025Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steeringcited by 24
  13. 2023Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learningcited by 22
  14. 2023Grokking Group Multiplication with Cosetscited by 22
  15. 2024Automatically Identifying Local and Global Circuits with Linear Computation Graphscited by 22
  16. 2024Controllable Context Sensitivity and the Knob Behind Itcited by 22
  17. 2025The Dual-Route Model of Inductioncited by 22
  18. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  19. 2026Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behaviorcited by 22
  20. 2025Causal Interventions Reveal Shared Structure Across English Filler-Gap Constructionscited by 21
  21. 2026There Will Be a Scientific Theory of Deep Learningcited by 21
  22. 2025How Do Transformers Learn Variable Binding in Symbolic Programs?cited by 19
  23. 2025How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencodingcited by 19
  24. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 18
  25. 2023Look Before You Leap: A Universal Emergent Decomposition of Retrieval Tasks in Language Modelscited by 17
  26. 2024A Philosophical Introduction to Language Models - Part II: The Way Forwardcited by 17
  27. 2024Efficiently Democratizing Medical LLMs for 50 Languages via a Mixture of Language Family Expertscited by 17
  28. 2026CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failurescited by 17
  29. 2024LLMs as Models for Analogical Reasoningcited by 16
  30. 2023Targeted Reduction of Causal Modelscited by 15
  31. 2025Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Modelscited by 14
  32. 2025Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEscited by 14
  33. 2024Gumbel Counterfactual Generation From Language Modelscited by 13
  34. 2025Evaluation Awareness Scales Predictably in Open-Weights Large Language Modelscited by 13
  35. 2025Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspectivecited by 12
  36. 2024Position: An Inner Interpretability Framework for AI Inspired by Lessons from Cognitive Neurosciencecited by 11
  37. 2025Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviorscited by 11
  38. 2025SALT: Steering Activations towards Leakage-free Thinking in Chain of Thoughtcited by 11

50 most cited of 194 All 194 (JSON)