mlsanityData to 14 Oct 2026

How Causal Abstraction Underpins Computational Explanation

arXiv 2508.11214 ↗First posted 15 Aug 2025cs.LG cs.AI cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

Explanations of cognitive behavior often appeal to computations over representations. What does it take for a system to implement a given computation over suitable representational vehicles within that system? We argue that the language of causality -- and specifically the theory of causal abstraction -- provides a fruitful lens on this topic. Drawing on current discussions in deep learning with artificial neural networks, we illustrate how classical themes in the philosophy of computation and cognition resurface in contemporary machine learning. We offer an account of computational implementation grounded in causal abstraction, and examine the role for representation in the resulting picture. We argue that these issues are most profitably explored in connection with generalization and prediction.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 56 of its 140 references that are in the map. First posted on arXiv 15 Aug 2025.

Builds on

56 of 140 references.

References · 56

  1. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  4. 2022Toy Models of Superpositioncited by 971
  5. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  6. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  7. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  8. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 571
  9. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  10. 2021Dynabench: Rethinking Benchmarking in NLPcited by 459
  11. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
  12. 2021Causal Abstractions of Neural Networkscited by 422
38 more
  1. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  2. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  3. 2023Function Vectors in Large Language Modelscited by 308
  4. 2013Direct and Indirect Effectscited by 280
  5. 2023Language Modeling Is Compressioncited by 225
  6. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
  7. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
  8. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
  9. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 202
  10. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 191
  11. 2016Foundations of Structural Causal Models with Cycles and Latent Variablescited by 134
  12. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 131
  13. 2017Causal Consistency of Structural Equation Modelscited by 118
  14. 2023How do Language Models Bind Entities in Context?cited by 108
  15. 2018Abstracting Causal Modelscited by 107
  16. 2023Interpretability at Scale: Identifying Causal Mechanisms in Alpacacited by 103
  17. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
  18. 2024Robust agents learn causal world modelscited by 81
  19. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 79
  20. 2022Polysemanticity and Capacity in Neural Networkscited by 77
  21. 2022Meaning without reference in large language modelscited by 76
  22. 2025MIB: A Mechanistic Interpretability Benchmarkcited by 56
  23. 2024Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representationscited by 47
  24. 2024CausalGym: Benchmarking causal interpretability methods on linguistic taskscited by 46
  25. 2025Language Models use Lookbacks to Track Beliefscited by 39
  26. 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?cited by 37
  27. 2023Discovering Variable Binding Circuitry with Desideratacited by 34
  28. 2023Deep neural networks have an inbuilt Occam's razorcited by 28
  29. 2024Representational Analysis of Binding in Language Modelscited by 28
  30. 2023A Geometric Notion of Causal Probingcited by 25
  31. 2024Mechanistic?cited by 25
  32. 2025Causal Interventions Reveal Shared Structure Across English Filler-Gap Constructionscited by 21
  33. 2020Relational reasoning and generalization using non-symbolic neural networkscited by 15
  34. 2022Abstraction between Structural Causal Models: A Review of Definitions and Propertiescited by 15
  35. 2023Operationalising Representation in Natural Language Processingcited by 14
  36. 2024Characterizing the Role of Similarity in the Property Inferences of Language Modelscited by 11
  37. 2025Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviorscited by 11
  38. 2022Beyond linear regression: mapping models in cognitive neuroscience should align with research goalscited by 10

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 5.

11 papers.

  1. 2025Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Contextcited by 26
  2. 2026Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Conceptscited by 10
  3. 2025Addressing divergent representations from causal interventions on neural networkscited by 7
  4. 2025Language models as tools for investigating the distinction between possible and impossible natural languagescited by 0
  5. 2025Large language models have learned to use languagecited by 0
  6. 2025Large language models are not about natural languagecited by 0
  7. 2025Linguists should learn to love speech-based deep learning modelscited by 0
  8. 2026Are Language Models Models?cited by 0
  9. 2026Across the Levels of Analysis: Explaining Predictive Processing in Humans Requires More Than Machine-Estimated Probabilitiescited by 0
  10. 2026You Can't Fight in Here! This is BBS!cited by 0
  11. 2026Not-So-Strange Love: Language Models and Generative Linguistic Theories are More Compatible than They Appearcited by 0