mlsanityData to 7 Oct 2026

(QA)2: Question Answering with Questionable Assumptions

arXiv 2212.10003 ↗First posted 20 Dec 2022cs.CLcited by 51

Not on the map. Why?

Abstract (arXiv)

Naturally occurring information-seeking questions often contain questionable assumptions -- assumptions that are false or unverifiable. Questions containing questionable assumptions are challenging because they require a distinct answer strategy that deviates from typical answers for information-seeking questions. For instance, the question "When did Marie Curie discover Uranium?" cannot be answered as a typical "when" question without addressing the false assumption "Marie Curie discovered Uranium". In this work, we propose (QA)2 (Question Answering with Questionable Assumptions), an open-domain evaluation dataset consisting of naturally occurring search engine queries that may or may not contain questionable assumptions. To be successful on (QA)2, systems must be able to detect questionable assumptions and also be able to produce adequate responses for both typical information-seeking questions and ones with questionable assumptions. Through human rater acceptability on end-to-end QA with (QA)2, we find that current models do struggle with handling questionable assumptions, leaving substantial headroom for progress.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Builds on 24 of its 48 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

24 of 48 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  5. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  6. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  7. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  8. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  9. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  10. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  11. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  12. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
12 more
  1. 2020Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESuppositioncited by 76
  2. 2022CREPE: Open-Domain Question Answering with False Presuppositionscited by 70
  3. 2021Can NLI Models Verify QA Systems' Predictions?cited by 46
  4. 2019Probing What Different NLP Tasks Teach Machines about Function Word Comprehensioncited by 44
  5. 2020Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrievalcited by 44
  6. 2021Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answeringcited by 41
  7. 2021General-Purpose Question-Answering with Macawcited by 40
  8. 2020"I'd rather just go to bed": Understanding Indirect Answerscited by 36
  9. 2021FaVIQ: FAct Verification from Information-seeking Questionscited by 36
  10. 2021NOPE: A Corpus of Naturally-Occurring Presuppositions in Englishcited by 23
  11. 2021He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on Pragmaticscited by 12
  12. 2018Let's do it "again": A First Computational Approach to Detecting Adverbial Presupposition Triggerscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 12; 2024: 15; 2025: 13; 2026: 11.

51 papers.

Sort:

By area:

  1. 2023How Language Model Hallucinations Can Snowballcited by 310
  2. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  3. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 129
  4. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
  5. 2023Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Modelscited by 60
  6. 2024PUB: A Pragmatics Understanding Benchmark for Assessing LLMs' Pragmatics Capabilitiescited by 58
  7. 2024Aligning Language Models to Explicitly Handle Ambiguitycited by 55
  8. 2023Personas as a Way to Model Truthfulness in Language Modelscited by 46
  9. 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Stepscited by 43
  10. 2023Enhancing Large Language Models Against Inductive Instructions with Dual-critique Promptingcited by 21
  11. 2025How to Protect Yourself from 5G Radiation? Investigating LLM Responses to Implicit Misinformationcited by 17
  12. 2025Cancer-Myth: Evaluating Large Language Models on Patient Questions with False Presuppositionscited by 15
38 more
  1. 2023Pregnant Questions: The Importance of Pragmatic Awareness in Maternal Health Question Answeringcited by 13
  2. 2024Reverse Question Answering: Can an LLM Write a Question so Hard (or Bad) that it Can't Answer?cited by 13
  3. 2025Towards LLM Agents for Earth Observationcited by 13
  4. 2023Evaluating Large Language Models for Health-related Queries with Presuppositionscited by 12
  5. 2025LLMs Struggle to Reject False Presuppositions when Misinformation Stakes are Highcited by 11
  6. 2026Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RLcited by 11
  7. 2026Over-Searching in Search-Augmented Large Language Modelscited by 10
  8. 2024Whispers that Shake Foundations: Analyzing and Mitigating False Premise Hallucinations in Large Language Modelscited by 9
  9. 2024KG-FPQ: Evaluating Factuality Hallucination in LLMs with Knowledge Graph-based False Premise Questionscited by 9
  10. 2025Can LLMs Ground when they (Don't) Know: A Study on Direct and Loaded Political Questionscited by 9
  11. 2026Agentic Abstention: Do Agents Know When to Stop Instead of Act?cited by 9
  12. 2025MultiHoax: A Dataset of Multi-hop False-Premise Questionscited by 8
  13. 2025RefusalBench: Generative Evaluation of Selective Refusal in Grounded Language Modelscited by 8
  14. 2023Asking More Informative Questions for Grounded Retrievalcited by 6
  15. 2024I Could've Asked That: Reformulating Unanswerable Questionscited by 6
  16. 2024Syn-QA2: Evaluating False Assumptions in Long-tail Questions with Synthetic QA Datasetscited by 5
  17. 2024DRS: Deep Question Reformulation With Structured Outputcited by 5
  18. 2023Can NLP Models 'Identify', 'Distinguish', and 'Justify' Questions that Don't have a Definitive Answer?cited by 4
  19. 2023PreWoMe: Exploiting Presuppositions as Working Memory for Long Form Question Answeringcited by 4
  20. 2024Unanswerability Evaluation for Retrieval Augmented Generationcited by 4
  21. 2024Towards Unbiased Evaluation of Detecting Unanswerable Questions in EHRSQLcited by 3
  22. 2024Red Teaming Language Models for Processing Contradictory Dialoguescited by 3
  23. 2025Knowing What's Missing: Assessing Information Sufficiency in Question Answeringcited by 3
  24. 2026Senses Wide Shut: A Representation-Action Gap in Omnimodal LLMscited by 2
  25. 2023Mastering the ABCDs of Complex Questions: Answer-Based Claim Decomposition for Fine-grained Self-Evaluationcited by 1
  26. 2024How the Advent of Ubiquitous Large Language Models both Stymie and Turbocharge Dynamic Adversarial Question Generationcited by 1
  27. 2024Interpreting Answers to Yes-No Questions in Dialogues from Multiple Domainscited by 1
  28. 2025FactGuard: Leveraging Multi-Agent Systems to Generate Answerable and Unanswerable Questions for Enhanced Long-Context LLM Extractioncited by 1
  29. 2025P-ReMIS: Pragmatic Reasoning in Mental Health and a Social Implicationcited by 1
  30. 2026HopRefusalBench: Diagnosing Refusal Failures in Search-Augmented Agents for Multi-Hop Reasoningcited by 1
  31. 2026Don't `Well, Actually' Me Unless You Know What You're Talking About: Weak Presupposition Verification Degrades General QA Performancecited by 1
  32. 2023A Lightweight Method to Generate Unanswerable Questions in Englishcited by 0
  33. 2024LIEDER: Linguistically-Informed Evaluation for Discourse Entity Recognitioncited by 0
  34. 2025Identifying and Answering Questions with False Assumptions: An Interpretable Approachcited by 0
  35. 2026iAgentBench: Benchmarking Sensemaking Capabilities of Information-Seeking Agents on High-Traffic Topicscited by 0
  36. 2026Towards Emotion Consistency Analysis of Large Language Models in Emotional Conversational Contextscited by 0
  37. 2026What's in a Name? Morphological Shortcuts by LLMs in Pharmacologycited by 0
  38. 2026Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase Itcited by 0

50 most cited of 51 All 51 (JSON)