mlsanityData to 7 Oct 2026

WikiWhy: Answering and Explaining Cause-and-Effect Questions

arXiv 2210.12152 ↗First posted 21 Oct 2022cs.CL cs.AIcited by 21

Not on the map. Why?

Abstract (arXiv)

As large language models (LLMs) grow larger and more sophisticated, assessing their "reasoning" capabilities in natural language grows more challenging. Recent question answering (QA) benchmarks that attempt to assess reasoning are often limited by a narrow scope of covered situations and subject matters. We introduce WikiWhy, a QA dataset built around a novel auxiliary task: explaining why an answer is true in natural language. WikiWhy contains over 9,000 "why" question-answer-rationale triples, grounded on Wikipedia facts across a diverse set of topics. Each rationale is a set of supporting statements connecting the question to the answer. WikiWhy serves as a benchmark for the reasoning capabilities of LLMs because it demands rigorous explicit rationales for each answer to demonstrate the acquisition of implicit commonsense knowledge, which is unlikely to be easily memorized. GPT-3 baselines achieve only 38.7% human-evaluated correctness in the end-to-end answer & explain condition, leaving significant room for future improvements.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 35 of its 47 references that are in the map. First posted on arXiv 21 Oct 2022.

Builds on

35 of 47 references.

References · 35

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  8. 2015VQA: Visual Question Answeringcited by 2,489
  9. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  10. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  11. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  12. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
23 more
  1. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  2. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  3. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  4. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  5. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  6. 2019ELI5: Long Form Question Answeringcited by 567
  7. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  8. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  9. 2022News Summarization and Evaluation in the Era of GPT-3cited by 324
  10. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 263
  11. 2022What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?cited by 149
  12. 2021Explaining Answers with Entailment Treescited by 148
  13. 2019Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answeringcited by 147
  14. 2020Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answeringcited by 49
  15. 2022The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoningcited by 38
  16. 2019R4C: A Benchmark for Evaluating RC Systems to Get the Right Answer for the Right Reasoncited by 36
  17. 2021TellMeWhy: A Dataset for Answering Why-Questions in Narrativescited by 27
  18. 2022Entailment Tree Explanations via Iterative Retrieval-Generation Reasonercited by 26
  19. 2021Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identitycited by 22
  20. 2022Towards Fine-grained Causal Reasoning and QAcited by 13
  21. 2021Re-evaluating Word Mover's Distancecited by 10
  22. 2021iReason: Multimodal Commonsense Reasoning using Videos and Natural Language with Interpretabilitycited by 4
  23. 2018Towards Inference-Oriented Reading Comprehension: ParallelQAcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 6; 2025: 7; 2026: 3.

21 papers.

Sort:

By area:

  1. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  2. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 146
  3. 2023CLadder: Assessing Causal Reasoning in Language Modelscited by 102
  4. 2025Measuring what Matters: Construct Validity in Large Language Model Benchmarkscited by 102
  5. 2024Instruction Pre-Training: Language Models are Supervised Multitask Learnerscited by 63
  6. 2023Natural Language Reasoning, A Surveycited by 61
  7. 2024Benchmarks as Microscopes: A Call for Model Metrologycited by 33
  8. 2024Causal Inference with Large Language Model: A Surveycited by 31
  9. 2023AutoKG: Efficient Automated Knowledge Graph Generation for Language Modelscited by 17
  10. 2025Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performancecited by 17
  11. 2024Failure Modes of LLMs for Causal Reasoning on Narrativescited by 12
  12. 2025MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibrationcited by 11
9 more
  1. 2024CHECKWHY: Causal Fact Verification via Argument Structurecited by 9
  2. 2025Counterfactual Simulatability of LLM Explanations for Generation Taskscited by 7
  3. 2024Lost in Translation? Translation Errors and Challenges for Fair Assessment of Text-to-Image Models on Multilingual Conceptscited by 5
  4. 2025Causal MAS: A Survey of Large Language Model Architectures for Discovery and Effect Estimationcited by 2
  5. 2025HugAgent: A Human Simulation Benchmark for Individual-Level Reasoningcited by 2
  6. 2025Natural Context Drift Undermines the Natural Language Understanding of Large Language Modelscited by 1
  7. 2026CommonWhy: A Dataset for Evaluating Entity-Based Causal Commonsense Reasoning in Large Language Modelscited by 0
  8. 2026The Illusion of textitWhat If: Evaluating the Breakdown of Counterfactual Reasoning in LLMscited by 0
  9. 2026Contextual Causality with Large Language Models: A Surveycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake
  2. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
  3. 2025CoLoTa: A Dataset for Entity-based Commonsense Reasoning over Long-Tail Knowledge
  4. 2024From Data to Commonsense Reasoning: The Use of Large Language Models for Explainable AI
  5. 2025Evaluating the Meta- and Object-Level Reasoning of Large Language Models for Question Answering
  6. 2025PhantomWiki: On-Demand Datasets for Reasoning and Retrieval Evaluation
  7. 2024LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models
  8. 2023ToolQA: A Dataset for LLM Question Answering with External Tools