# Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering

arXiv: https://arxiv.org/abs/2101.00391 · HTML: https://mlsanity.com/paper/2101.00391/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 41 references that are in the map.
- First posted on arXiv 2 Jan 2021.

## Builds on (most cited first, 18 of 18)

- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [ELI5: Long Form Question Answering](https://mlsanity.com/paper/1907.09190/) (2019)
- [AmbigQA: Answering Ambiguous Open-domain Questions](https://mlsanity.com/paper/2004.10645/) (2020)
- [Simple and Effective Multi-Paragraph Reading Comprehension](https://mlsanity.com/paper/1710.10723/) (2017)
- [Fine-grained Fact Verification with Kernel Graph Attention Network](https://mlsanity.com/paper/1910.09796/) (2019)
- [Retrospective Reader for Machine Reading Comprehension](https://mlsanity.com/paper/2001.09694/) (2020)
- [Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition](https://mlsanity.com/paper/2004.03066/) (2020)
- [QED: A Framework and Dataset for Explanations in Question Answering](https://mlsanity.com/paper/2009.06354/) (2020)
- [Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrieval](https://mlsanity.com/paper/2010.11915/) (2020)
- [Read + Verify: Machine Reading Comprehension with Unanswerable Questions](https://mlsanity.com/paper/1808.05759/) (2018)
- [A Qualitative Comparison of CoQA, SQuAD 2.0 and QuAC](https://mlsanity.com/paper/1809.10735/) (2018)
- [Learning to Ask Unanswerable Questions for Machine Reading Comprehension](https://mlsanity.com/paper/1906.06045/) (2019)
- [Let's do it "again": A First Computational Approach to Detecting Adverbial Presupposition Triggers](https://mlsanity.com/paper/1806.04262/) (2018)

## Built on by (most cited first, 30 of 41)

- [How Language Model Hallucinations Can Snowball](https://mlsanity.com/paper/2305.13534/) (2023)
- [SituatedQA: Incorporating Extra-Linguistic Contexts into QA](https://mlsanity.com/paper/2109.06157/) (2021)
- [DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering](https://mlsanity.com/paper/2211.05655/) (2022)
- [Few-Shot Self-Rationalization with Natural Language Prompts](https://mlsanity.com/paper/2111.08284/) (2021)
- [CREPE: Open-Domain Question Answering with False Presuppositions](https://mlsanity.com/paper/2211.17257/) (2022)
- [PUB: A Pragmatics Understanding Benchmark for Assessing LLMs' Pragmatics Capabilities](https://mlsanity.com/paper/2401.07078/) (2024)
- [(QA)2: Question Answering with Questionable Assumptions](https://mlsanity.com/paper/2212.10003/) (2022)
- [Won't Get Fooled Again: Answering Questions with False Premises](https://mlsanity.com/paper/2307.02394/) (2023)
- [Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps](https://mlsanity.com/paper/2502.14829/) (2025)
- [Can NLI Models Verify QA Systems' Predictions?](https://mlsanity.com/paper/2104.08731/) (2021)
- [General-Purpose Question-Answering with Macaw](https://mlsanity.com/paper/2109.02593/) (2021)
- [FaVIQ: FAct Verification from Information-seeking Questions](https://mlsanity.com/paper/2107.02153/) (2021)
- [NOPE: A Corpus of Naturally-Occurring Presuppositions in English](https://mlsanity.com/paper/2109.06987/) (2021)
- [Enhancing Large Language Models Against Inductive Instructions with Dual-critique Prompting](https://mlsanity.com/paper/2305.13733/) (2023)
- [How Do We Answer Complex Questions: Discourse Structure of Long-form Answers](https://mlsanity.com/paper/2203.11048/) (2022)
- [Don't Let It Hallucinate: Premise Verification via Retrieval-Augmented Logical Reasoning](https://mlsanity.com/paper/2504.06438/) (2025)
- [Pregnant Questions: The Importance of Pragmatic Awareness in Maternal Health Question Answering](https://mlsanity.com/paper/2311.09542/) (2023)
- [LLMs Struggle to Reject False Presuppositions when Misinformation Stakes are High](https://mlsanity.com/paper/2505.22354/) (2025)
- [Large Language Model Programs](https://mlsanity.com/paper/2305.05364/) (2023)
- [Can LLMs Ground when they (Don't) Know: A Study on Direct and Loaded Political Questions](https://mlsanity.com/paper/2506.08952/) (2025)
- [SCENE: Self-Labeled Counterfactuals for Extrapolating to Negative Examples](https://mlsanity.com/paper/2305.07984/) (2023)
- [Don't Take the Premise for Granted: Evaluating the Premise Critique Ability of Large Language Models](https://mlsanity.com/paper/2505.23715/) (2025)
- [EpiK-Eval: Evaluation for Language Models as Epistemic Models](https://mlsanity.com/paper/2310.15372/) (2023)
- [Asking More Informative Questions for Grounded Retrieval](https://mlsanity.com/paper/2311.08584/) (2023)
- [I Could've Asked That: Reformulating Unanswerable Questions](https://mlsanity.com/paper/2407.17469/) (2024)
- [Syn-QA2: Evaluating False Assumptions in Long-tail Questions with Synthetic QA Datasets](https://mlsanity.com/paper/2403.12145/) (2024)
- [If We May De-Presuppose: Robustly Verifying Claims through Presupposition-Free Question Decomposition](https://mlsanity.com/paper/2508.16838/) (2025)
- [Can NLP Models 'Identify', 'Distinguish', and 'Justify' Questions that Don't have a Definitive Answer?](https://mlsanity.com/paper/2309.04635/) (2023)
- [PreWoMe: Exploiting Presuppositions as Working Memory for Long Form Question Answering](https://mlsanity.com/paper/2310.16147/) (2023)
- [Answering Unanswered Questions through Semantic Reformulations in Spoken QA](https://mlsanity.com/paper/2305.17393/) (2023)

## Abstract (arXiv)

Many Question-Answering (QA) datasets contain unanswerable questions, but their treatment in QA systems remains primitive. Our analysis of the Natural Questions (Kwiatkowski et al. 2019) dataset reveals that a substantial portion of unanswerable questions (~21%) can be explained based on the presence of unverifiable presuppositions. We discuss the shortcomings of current models in handling such questions, and describe how an improved system could handle them. Through a user preference study, we demonstrate that the oracle behavior of our proposed system that provides responses based on presupposition failure is preferred over the oracle behavior of existing QA systems. Then we discuss how our proposed system could be implemented, presenting a novel framework that breaks down the problem into three steps: presupposition generation, presupposition verification and explanation generation. We report our progress in tackling each subproblem, and present a preliminary approach to integrating these steps into an existing QA system. We find that adding presuppositions and their verifiability to an existing model yields modest gains in downstream performance and unanswerability detection. The biggest bottleneck is the verification component, which needs to be substantially improved for the integrated system to approach ideal behavior -- even transfer from the best entailment models currently falls short.