mlsanityData to 14 Oct 2026

Causal interventions expose implicit situation models for commonsense language understanding

arXiv 2306.03882 ↗First posted 6 Jun 2023cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Accounts of human language processing have long appealed to implicit ``situation models'' that enrich comprehension with relevant but unstated world knowledge. Here, we apply causal intervention techniques to recent transformer models to analyze performance on the Winograd Schema Challenge (WSC), where a single context cue shifts interpretation of an ambiguous pronoun. We identify a relatively small circuit of attention heads that are responsible for propagating information from the context word that guides which of the candidate noun phrases the pronoun ultimately attends to. We then compare how this circuit behaves in a closely matched ``syntactic'' control where the situation model is not strictly necessary. These analyses suggest distinct pathways through which implicit situation models are constructed to guide pronoun resolution.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 41 of its 72 references that are in the map. First posted on arXiv 6 Jun 2023.

Builds on

41 of 72 references.

References · 41

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  7. 2019Language Models as Knowledge Bases?cited by 1,936
  8. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  9. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  10. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  11. 2022In-context Learning and Induction Headscited by 964
  12. 2021Knowledge Neurons in Pretrained Transformerscited by 601
29 more
  1. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  2. 2021Causal Abstractions of Neural Networkscited by 422
  3. 2019Revealing the Dark Secrets of BERTcited by 301
  4. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
  5. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  6. 2021Implicit Representations of Meaning in Neural Language Modelscited by 180
  7. 2020Probing Pretrained Language Models for Lexical Semanticscited by 156
  8. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 131
  9. 2023Interpretability at Scale: Identifying Causal Mechanisms in Alpacacited by 103
  10. 2020Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Maskscited by 98
  11. 2022Interpreting Language Models with Contrastive Explanationscited by 92
  12. 2022Meaning without reference in large language modelscited by 76
  13. 2021Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Predictioncited by 59
  14. 2022Probing for the Usage of Grammatical Numbercited by 59
  15. 2023Quantifying Context Mixing in Transformerscited by 43
  16. 2020Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humanscited by 42
  17. 2019Discovering the Compositional Structure of Vector Representations with Role Learning Networkscited by 34
  18. 2021Sparse Interventions in Language Models with Differentiable Maskingcited by 34
  19. 2021Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schemacited by 32
  20. 2021Relational World Knowledge Representation in Contextual Language Models: A Reviewcited by 31
  21. 2021Combining pre-trained language models and structured knowledgecited by 25
  22. 2022The Defeat of the Winograd Schema Challengecited by 25
  23. 2020The Sensitivity of Language Models and Humans to Winograd Schema Perturbationscited by 21
  24. 2018How Reasonable are Common-Sense Reasoning Tasks: A Case-Study on the Winograd Schema Challenge and SWAGcited by 18
  25. 2020Discourse structure interacts with reference but not syntax in neural language modelscited by 17
  26. 2022Inducing Character-level Structure in Subword-based Language Models with Type-level Interchange Intervention Trainingcited by 16
  27. 2022Causal Analysis of Syntactic Agreement Neurons in Multilingual Language Modelscited by 14
  28. 2020Precise Task Formalization Matters in Winograd Schema Evaluationscited by 7
  29. 2022Language Modeling with Latent Situationscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 0; 2026: 2.

7 papers.

  1. 2024ReFT: Representation Finetuning for Language Modelscited by 197
  2. 2024CausalGym: Benchmarking causal interpretability methods on linguistic taskscited by 46
  3. 2024On the Tip of the Tongue: Analyzing Conceptual Representation in Large Language Models with Reverse-Dictionary Probecited by 6
  4. 2024GPT-ology, Computational Models, Silicon Sampling: How should we think about LLMs in Cognitive Science?cited by 5
  5. 2024Perception of Phonological Assimilation by Neural Speech Recognition Modelscited by 3
  6. 2026Indications of Belief-Guided Agency and Meta-Cognitive Monitoring in Large Language Modelscited by 3
  7. 2026The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Modelscited by 1