mlsanityData to 7 Oct 2026

Implicit causality in GPT-2: a case study

arXiv 2212.04348 ↗First posted 8 Dec 2022cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

This case study investigates the extent to which a language model (GPT-2) is able to capture native speakers' intuitions about implicit causality in a sentence completion task. We first reproduce earlier results (showing lower surprisal values for pronouns that are congruent with either the subject or object, depending on which one corresponds to the implicit causality bias of the verb), and then examine the effects of gender and verb frequency on model performance. Our second study examines the reasoning ability of GPT-2: is the model able to produce more sensible motivations for why the subject VERBed the object if the verbs have stronger causality biases? We also developed a methodology to avoid human raters being biased by obscenities and disfluencies generated by the model.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 10 of its 31 references that are in the map. First posted on arXiv 8 Dec 2022.

Builds on

10 of 31 references.

References · 10

  1. 2021Ethical and social risks of harm from Language Modelscited by 957
  2. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  3. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  4. 2019Unifying Human and Statistical Evaluation for Natural Language Generationcited by 127
  5. 2020Trading Off Diversity and Quality in Natural Language Generationcited by 94
  6. 2020A Theoretical Analysis of the Repetition Problem in Text Generationcited by 83
  7. 2021Underreporting of errors in NLG output, and what to do about itcited by 19
  8. 2020Discourse structure interacts with reference but not syntax in neural language modelscited by 17
  9. 2022Machine Reading, Fast and Slow: When Do Models "Understand" Language?cited by 13
  10. 2021Uncovering Constraint-Based Behavior in Neural Models via Targeted Fine-Tuningcited by 5

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 1.

2 papers.

  1. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  2. 2025Implicit Causality-biases in humans and LLMs as a tool for benchmarking LLM discourse capabilitiescited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Causal Interventions on Causal Paths: Mapping GPT-2's Reasoning From Syntax to Semantics
  2. 2025Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
  3. 2025CLEAR-3K: Assessing Causal Explanatory Capabilities in Language Models
  4. 2023CLadder: Assessing Causal Reasoning in Language Models
  5. 2023Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluation
  6. 2022Underspecification in Language Modeling Tasks: A Causality-Informed Study of Gendered Pronoun Resolution
  7. 2026Causal Tongue-Tie: LLMs Can Encode Causal Direction, But Their Yes/No Outputs Fail to Express
  8. 2024A Critical Review of Causal Reasoning Benchmarks for Large Language Models