# Counterfactual reasoning: Testing language models' understanding of hypothetical scenarios

arXiv: https://arxiv.org/abs/2305.16572 · HTML: https://mlsanity.com/paper/2305.16572/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Causal reasoning in LLMs, one of 17 directions in LLM reasoning.
- Builds on 10 of its 21 references that are in the map.
- First posted on arXiv 26 May 2023.

## Placed in

- LLM reasoning / [Causal reasoning in LLMs](https://mlsanity.com/lr/causal-reasoning-in-llms/)

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Explain Yourself! Leveraging Language Models for Commonsense Reasoning](https://mlsanity.com/paper/1906.02361/) (2019)
- [Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought](https://mlsanity.com/paper/2210.01240/) (2022)
- [Probing Neural Network Comprehension of Natural Language Arguments](https://mlsanity.com/paper/1907.07355/) (2019)
- [Counterfactual Story Reasoning and Generation](https://mlsanity.com/paper/1909.04076/) (2019)
- [CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Models](https://mlsanity.com/paper/2112.11941/) (2021)

## Built on by (most cited first, 27 of 27)

- [Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks](https://mlsanity.com/paper/2307.02477/) (2023)
- [Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey](https://mlsanity.com/paper/2404.01869/) (2024)
- [Characterizing Mechanisms for Factual Recall in Language Models](https://mlsanity.com/paper/2310.15910/) (2023)
- [Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?](https://mlsanity.com/paper/2506.21215/) (2025)
- [Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledge](https://mlsanity.com/paper/2311.09731/) (2023)
- [Controllable Context Sensitivity and the Knob Behind It](https://mlsanity.com/paper/2411.07404/) (2024)
- [MenatQA: A New Dataset for Testing the Temporal Comprehension and Reasoning Abilities of Large Language Models](https://mlsanity.com/paper/2310.05157/) (2023)
- [A Theory for Length Generalization in Learning to Reason](https://mlsanity.com/paper/2404.00560/) (2024)
- [MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset](https://mlsanity.com/paper/2406.02106/) (2024)
- [Conditions for Length Generalization in Learning Reasoning Skills](https://mlsanity.com/paper/2311.16173/) (2023)
- [Pre-Trained Language Models Represent Some Geographic Populations Better Than Others](https://mlsanity.com/paper/2403.11025/) (2024)
- [The Odyssey of Commonsense Causality: From Foundational Benchmarks to Cutting-Edge Reasoning](https://mlsanity.com/paper/2406.19307/) (2024)
- [On the Eligibility of LLMs for Counterfactual Reasoning: A Decompositional Study](https://mlsanity.com/paper/2505.11839/) (2025)
- [Counterfactual reasoning: an analysis of in-context emergence](https://mlsanity.com/paper/2506.05188/) (2025)
- [GraphLocator: Graph-guided Causal Reasoning for Issue Localization](https://mlsanity.com/paper/2512.22469/) (2025)
- [The Base-Rate Effect on LLM Benchmark Performance: Disambiguating Test-Taking Strategies from Benchmark Performance](https://mlsanity.com/paper/2406.11634/) (2024)
- [Exploring the Hidden Reasoning Process of Large Language Models by Misleading Them](https://mlsanity.com/paper/2503.16401/) (2025)
- [CaSKG: Counterfactual-Causal Skill Graphs for Scalable Agent Skill Retrieval](https://mlsanity.com/paper/2608.25500/) (2026)
- [Evidence-Grounded Verified Agentic Reasoning: A Path Toward Eliminating LLM Hallucination in Empirical Inference via Tool-Attested Kernel Proofs](https://mlsanity.com/paper/2607.12650/) (2026)
- [Counterfactual Reasoning with Knowledge Graph Embeddings](https://mlsanity.com/paper/2403.06936/) (2024)
- [ACCORD: Closing the Commonsense Measurability Gap](https://mlsanity.com/paper/2406.02804/) (2024)
- [Robustness of Neurosymbolic Reasoners on First-Order Logic Problems](https://mlsanity.com/paper/2509.17377/) (2025)
- [Catch Me If You Can: How Smaller Reasoning Models Pretend to Reason with Mathematical Fidelity](https://mlsanity.com/paper/2512.00552/) (2025)
- [Evaluating LLMs as Human Surrogates in Controlled Experiments](https://mlsanity.com/paper/2604.15329/) (2026)
- [CauSim: Scaling Causal Reasoning with Increasingly Complex Causal Simulators](https://mlsanity.com/paper/2605.09079/) (2026)
- [One mechanism for many mental spaces: a shared router over a value slot in language models](https://mlsanity.com/paper/2607.10248/) (2026)
- [Can AI Scientists Change Their Minds? Prior-Evidence Conflict in Synthetic Universes](https://mlsanity.com/paper/2609.36726/) (2026)

## Abstract (arXiv)

Current pre-trained language models have enabled remarkable improvements in downstream tasks, but it remains difficult to distinguish effects of statistical correlation from more systematic logical reasoning grounded on the understanding of real world. We tease these factors apart by leveraging counterfactual conditionals, which force language models to predict unusual consequences based on hypothetical propositions. We introduce a set of tests from psycholinguistic experiments, as well as larger-scale controlled datasets, to probe counterfactual predictions from five pre-trained language models. We find that models are consistently able to override real-world knowledge in counterfactual scenarios, and that this effect is more robust in case of stronger baseline world knowledge -- however, we also find that for most models this effect appears largely to be driven by simple lexical cues. When we mitigate effects of both world knowledge and lexical cues to test knowledge of linguistic nuances of counterfactuals, we find that only GPT-3 shows sensitivity to these nuances, though this sensitivity is also non-trivially impacted by lexical associative factors.