# WIQA: A dataset for "What if..." reasoning over procedural text

arXiv: https://arxiv.org/abs/1909.04739 · HTML: https://mlsanity.com/paper/1909.04739/

## Key facts

- Cited by 71 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Sep 2019.

## Built on by (most cited first, 30 of 71)

- [MetaICL: Learning to Learn In Context](https://mlsanity.com/paper/2110.15943/) (2021)
- [Can Language Models Solve Graph Problems in Natural Language?](https://mlsanity.com/paper/2305.10037/) (2023)
- [Language Models of Code are Few-Shot Commonsense Learners](https://mlsanity.com/paper/2210.07128/) (2022)
- [CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP](https://mlsanity.com/paper/2104.08835/) (2021)
- [Exploring the Benefits of Training Expert Language Models over Instruction Tuning](https://mlsanity.com/paper/2302.03202/) (2023)
- [Natural Language Reasoning, A Survey](https://mlsanity.com/paper/2303.14725/) (2023)
- [Pre-Training to Learn in Context](https://mlsanity.com/paper/2305.09137/) (2023)
- [Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models](https://mlsanity.com/paper/2310.00836/) (2023)
- [DAGN: Discourse-Aware Graph Network for Logical Reasoning](https://mlsanity.com/paper/2103.14349/) (2021)
- [IfQA: A Dataset for Open-domain Question Answering under Counterfactual Presuppositions](https://mlsanity.com/paper/2305.14010/) (2023)
- [Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models](https://mlsanity.com/paper/2210.16433/) (2022)
- [Causal Reasoning of Entities and Events in Procedural Texts](https://mlsanity.com/paper/2301.10896/) (2023)
- [Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data](https://mlsanity.com/paper/2203.07264/) (2022)
- [Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners](https://mlsanity.com/paper/2210.02969/) (2022)
- [Think about it! Improving defeasible reasoning by first modeling the question scenario](https://mlsanity.com/paper/2110.12349/) (2021)
- [Classifying Argumentative Relations Using Logical Mechanisms and Argumentation Schemes](https://mlsanity.com/paper/2105.07571/) (2021)
- [Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalization](https://mlsanity.com/paper/2210.09175/) (2022)
- [CLEVR_HYP: A Challenge Dataset and Baselines for Visual Question Answering with Hypothetical Actions over Images](https://mlsanity.com/paper/2104.05981/) (2021)
- [Task Compass: Scaling Multi-task Pre-training with Task Prefix](https://mlsanity.com/paper/2210.06277/) (2022)
- [Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective](https://mlsanity.com/paper/2210.08590/) (2022)
- [Mitigating Hallucinations in Large Language Models via Causal Reasoning](https://mlsanity.com/paper/2508.12495/) (2025)
- [On the Influence of Masking Policies in Intermediate Pre-training](https://mlsanity.com/paper/2104.08840/) (2021)
- [DomiKnowS: A Library for Integration of Symbolic Domain Knowledge in Deep Learning](https://mlsanity.com/paper/2108.12370/) (2021)
- [Eliciting and Understanding Cross-Task Skills with Task-Level Mixture-of-Experts](https://mlsanity.com/paper/2205.12701/) (2022)
- [Learning to Initialize: Can Meta Learning Improve Cross-task Generalization in Prompt Tuning?](https://mlsanity.com/paper/2302.08143/) (2023)
- [Beyond Graphs: Can Large Language Models Comprehend Hypergraphs?](https://mlsanity.com/paper/2410.10083/) (2024)
- [Could you give me a hint? Generating inference graphs for defeasible reasoning](https://mlsanity.com/paper/2105.05418/) (2021)
- [Reasoning about Actions over Visual and Linguistic Modalities: A Survey](https://mlsanity.com/paper/2207.07568/) (2022)
- [Executable Counterfactuals: Improving LLMs' Causal Reasoning Through Code](https://mlsanity.com/paper/2510.01539/) (2025)
- [PaCo: Preconditions Attributed to Commonsense Knowledge](https://mlsanity.com/paper/2104.08712/) (2021)

## Abstract (arXiv)

We introduce WIQA, the first large-scale dataset of "What if..." questions over procedural text. WIQA contains three parts: a collection of paragraphs each describing a process, e.g., beach erosion; a set of crowdsourced influence graphs for each paragraph, describing how one change affects another; and a large (40k) collection of "What if...?" multiple-choice questions derived from the graphs. For example, given a paragraph about beach erosion, would stormy weather result in more or less erosion (or have no effect)? The task is to answer the questions, given their associated paragraph. WIQA contains three kinds of questions: perturbations to steps mentioned in the paragraph; external (out-of-paragraph) perturbations requiring commonsense knowledge; and irrelevant (no effect) perturbations. We find that state-of-the-art models achieve 73.8% accuracy, well below the human performance of 96.3%. We analyze the challenges, in particular tracking chains of influences, and present the dataset as an open challenge to the community.