mlsanityData to 7 Oct 2026

WIQA: A dataset for "What if..." reasoning over procedural text

arXiv 1909.04739 ↗First posted 10 Sep 2019cs.CL cs.AIcited by 71

Not on the map. Why?

Abstract (arXiv)

We introduce WIQA, the first large-scale dataset of "What if..." questions over procedural text. WIQA contains three parts: a collection of paragraphs each describing a process, e.g., beach erosion; a set of crowdsourced influence graphs for each paragraph, describing how one change affects another; and a large (40k) collection of "What if...?" multiple-choice questions derived from the graphs. For example, given a paragraph about beach erosion, would stormy weather result in more or less erosion (or have no effect)? The task is to answer the questions, given their associated paragraph. WIQA contains three kinds of questions: perturbations to steps mentioned in the paragraph; external (out-of-paragraph) perturbations requiring commonsense knowledge; and irrelevant (no effect) perturbations. We find that state-of-the-art models achieve 73.8% accuracy, well below the human performance of 96.3%. We analyze the challenges, in particular tracking chains of influences, and present the dataset as an open challenge to the community.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 23; 2023: 16; 2024: 7; 2025: 9; 2026: 2.

71 papers.

Sort:

By area:

  1. 2021MetaICL: Learning to Learn In Contextcited by 450
  2. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
  3. 2022Language Models of Code are Few-Shot Commonsense Learnerscited by 160
  4. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  5. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  6. 2023Natural Language Reasoning, A Surveycited by 61
  7. 2023Pre-Training to Learn in Contextcited by 41
  8. 2023Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Modelscited by 35
  9. 2021DAGN: Discourse-Aware Graph Network for Logical Reasoningcited by 28
  10. 2023IfQA: A Dataset for Open-domain Question Answering under Counterfactual Presuppositionscited by 25
  11. 2022Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Modelscited by 24
  12. 2023Causal Reasoning of Entities and Events in Procedural Textscited by 23
38 more
  1. 2022Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Datacited by 22
  2. 2022Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learnerscited by 22
  3. 2021Think about it! Improving defeasible reasoning by first modeling the question scenariocited by 20
  4. 2021Classifying Argumentative Relations Using Logical Mechanisms and Argumentation Schemescited by 17
  5. 2022Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalizationcited by 15
  6. 2021CLEVR_HYP: A Challenge Dataset and Baselines for Visual Question Answering with Hypothetical Actions over Imagescited by 14
  7. 2022Task Compass: Scaling Multi-task Pre-training with Task Prefixcited by 14
  8. 2022Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspectivecited by 13
  9. 2025Mitigating Hallucinations in Large Language Models via Causal Reasoningcited by 13
  10. 2021On the Influence of Masking Policies in Intermediate Pre-trainingcited by 12
  11. 2021DomiKnowS: A Library for Integration of Symbolic Domain Knowledge in Deep Learningcited by 12
  12. 2022Eliciting and Understanding Cross-Task Skills with Task-Level Mixture-of-Expertscited by 12
  13. 2023Learning to Initialize: Can Meta Learning Improve Cross-task Generalization in Prompt Tuning?cited by 11
  14. 2024Beyond Graphs: Can Large Language Models Comprehend Hypergraphs?cited by 11
  15. 2021Could you give me a hint? Generating inference graphs for defeasible reasoningcited by 10
  16. 2022Reasoning about Actions over Visual and Linguistic Modalities: A Surveycited by 10
  17. 2025Executable Counterfactuals: Improving LLMs' Causal Reasoning Through Codecited by 10
  18. 2021PaCo: Preconditions Attributed to Commonsense Knowledgecited by 9
  19. 2023GLUECons: A Generic Benchmark for Learning Under Constraintscited by 9
  20. 2025GraphThought: Graph Combinatorial Optimization with Thought Generationcited by 9
  21. 2022Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learningcited by 8
  22. 2022Reasoning about Procedures with Natural Language Processing: A Tutorialcited by 8
  23. 2022Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Taskscited by 8
  24. 2023OpenPI2.0: An Improved Dataset for Entity Tracking in Textscited by 8
  25. 2024Chain and Causal Attention for Efficient Entity Trackingcited by 8
  26. 2023Teaching Probabilistic Logical Reasoning to Transformerscited by 7
  27. 2023CLOMO: Counterfactual Logical Modification with Large Language Modelscited by 7
  28. 2022Few-shot Adaptation Works with UnpredicTable Datacited by 6
  29. 2021Improving Neural Model Performance through Natural Language Feedback on Their Explanationscited by 5
  30. 2023Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learningcited by 5
  31. 2025Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applicationscited by 5
  32. 2021Relational Gating for "What If" Reasoningcited by 4
  33. 2022Learning Action Conditions from Instructional Manuals for Instruction Understandingcited by 4
  34. 2022PASTA: A Dataset for Modeling Participant States in Narrativescited by 4
  35. 2023The Role of Semantic Parsing in Understanding Procedural Textcited by 4
  36. 2025Graph-Grounded LLMs: Leveraging Graphical Function Calling to Minimize LLM Hallucinationscited by 4
  37. 2025Counterfactual reasoning: an analysis of in-context emergencecited by 4
  38. 2021CURIE: An Iterative Querying Approach for Reasoning About Situationscited by 3

50 most cited of 71 All 71 (JSON)