mlsanityData to 14 Oct 2026

CausalGym: Benchmarking causal interpretability methods on linguistic tasks

arXiv 2402.12560 ↗First posted 19 Feb 2024cs.CL cs.AIcited by 46

Not on the map. Why?

Abstract (arXiv)

Language models (LMs) have proven to be powerful tools for psycholinguistic research, but most prior work has focused on purely behavioural measures (e.g., surprisal comparisons). At the same time, research in model interpretability has begun to illuminate the abstract causal mechanisms shaping LM behavior. To help bring these strands of research closer together, we introduce CausalGym. We adapt and expand the SyntaxGym suite of tasks to benchmark the ability of interpretability methods to causally affect model behaviour. To illustrate how CausalGym can be used, we study the pythia models (14M--6.9B) and assess the causal efficacy of a wide range of interpretability methods, including linear probing and distributed alignment search (DAS). We find that DAS outperforms the other methods, and so we use it to study the learning trajectory of two difficult linguistic phenomena in pythia-1b: negative polarity item licensing and filler--gap dependencies. Our analysis shows that the mechanism implementing both of these tasks is learned in discrete stages, not gradually.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). Builds on 36 of its 58 references that are in the map. First posted on arXiv 19 Feb 2024.

Builds on

36 of 58 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  4. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  5. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  7. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  8. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  9. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  10. 2019Designing and Interpreting Probes with Control Taskscited by 640
  11. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
  12. 2021Causal Abstractions of Neural Networkscited by 422
24 more
  1. 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 402
  2. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 354
  3. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  4. 2023LEACE: Perfect linear concept erasure in closed formcited by 267
  5. 2018Targeted Syntactic Evaluation of Language Modelscited by 242
  6. 2016Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Taskscited by 216
  7. 2018Colorless green recurrent networks dream hierarchicallycited by 213
  8. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
  9. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
  10. 2023Localizing Model Behavior with Path Patchingcited by 195
  11. 2020A Systematic Assessment of Syntactic Generalization in Neural Language Modelscited by 165
  12. 2019Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic Statecited by 138
  13. 2023Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMscited by 124
  14. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 110
  15. 2023Interpretability at Scale: Identifying Causal Mechanisms in Alpacacited by 103
  16. 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 99
  17. 2022Probing for the Usage of Grammatical Numbercited by 59
  18. 2024Language models align with human judgments on key grammatical constructionscited by 49
  19. 2023A Geometric Notion of Causal Probingcited by 25
  20. 2023Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Modelcited by 22
  21. 2022Naturalistic Causal Probing for Morpho-Syntaxcited by 19
  22. 2023When Language Models Fall in Love: Animacy Processing in Transformer Language Modelscited by 12
  23. 2023Verb Conjugation in Transformers Is Determined by Linear Encodings of Subject Numbercited by 12
  24. 2023Causal interventions expose implicit situation models for commonsense language understandingcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 14; 2025: 12; 2026: 20.

46 papers.

Sort:

By area:

  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  2. 2024ReFT: Representation Finetuning for Language Modelscited by 197
  3. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
  4. 2025Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languagescited by 59
  5. 2025MIB: A Mechanistic Interpretability Benchmarkcited by 56
  6. 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?cited by 37
  7. 2026Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behaviorcited by 26
  8. 2024InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniquescited by 25
  9. 2024Mechanistic?cited by 25
  10. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 25
  11. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
  12. 2024On the Similarity of Circuits across Languages: a Case Study on the Subject-verb Agreement Taskcited by 24
34 more
  1. 2025Causal Interventions Reveal Shared Structure Across English Filler-Gap Constructionscited by 21
  2. 2024A Psycholinguistic Evaluation of Language Models' Sensitivity to Argument Rolescited by 14
  3. 2024A Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinesecited by 13
  4. 2026Steering Vector Fields for Context-Aware Inference-Time Control in Large Language Modelscited by 13
  5. 2024How Reliable are Causal Probing Interventions?cited by 11
  6. 2024Characterizing the Role of Similarity in the Property Inferences of Language Modelscited by 11
  7. 2025A Close Look at Decomposition-based XAI-Methods for Transformer Language Modelscited by 11
  8. 2025Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviorscited by 11
  9. 2025How Causal Abstraction Underpins Computational Explanationcited by 11
  10. 2025Mechanisms vs. Outcomes: Probing for Syntax Fails to Explain Performance on Targeted Syntactic Evaluationscited by 7
  11. 2025Analyzing the Inner Workings of Transformers in Compositional Generalizationcited by 6
  12. 2025Learning Distribution-Wise Control in Representation Space for Language Modelscited by 4
  13. 2026Pando: Do Interpretability Methods Work When Models Won't Explain Themselves?cited by 4
  14. 2024Challenges and Opportunities in Text Generation Explainabilitycited by 2
  15. 2025Steering Language Models in Multi-Token Generation: A Case Study on Tense and Aspectcited by 2
  16. 2026Humans and transformer LMs: Abstraction drives language learningcited by 2
  17. 2024Using Shapley interactions to understand how models use structurecited by 1
  18. 2024Linguistic Minimal Pairs Elicit Linguistic Similarity in Large Language Modelscited by 1
  19. 2026The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Modelscited by 1
  20. 2026Perturbation: A simple and efficient adversarial tracer for representation learning in language modelscited by 1
  21. 2026Bucketing the Good Apples: A Method for Diagnosing and Improving Causal Abstractioncited by 1
  22. 2025Investigating Syntactic Biases in Multilingual Transformers with RC Attachment Ambiguities in Italian and Englishcited by 0
  23. 2026Enhancing Structured Meaning Representations with Aspect Classificationcited by 0
  24. 2026Causal Drawbridges: Characterizing Gradient Blocking of Syntactic Islands in Transformer LMscited by 0
  25. 2026Filling in the Mechanisms: How do LMs Learn Filler-Gap Dependencies under Developmental Constraints?cited by 0
  26. 2026Fine-Grained Analysis of Shared Syntactic Mechanisms in Language Modelscited by 0
  27. 2026Inference Time Causal Probing in LLMscited by 0
  28. 2026When Context Misleads: Surprisal, Energy and Attention Entropy as Metrics of Coherence Illusions in LLMscited by 0
  29. 2026Causal Interventions Reveal Typologically Organized Syntactic Mechanisms in Multilingual Language Modelscited by 0
  30. 2026(V)LMs generalize beyond surface co-occurrence: Evidence from cross-modal number agreementcited by 0
  31. 2026Tracing Stereotypes from Representation to Output in Multilingual LLMscited by 0
  32. 2026Direct Hidden-State Alignment: Mapping and Controlling Preference Expression in LLMscited by 0
  33. 2026Causal and Interpretable Structures in LLM Compositional Taskscited by 0
  34. 2026From Copying to Composition: How Language Models Develop In-Context Learningcited by 0