# Won't Get Fooled Again: Answering Questions with False Premises

arXiv: https://arxiv.org/abs/2307.02394 · HTML: https://mlsanity.com/paper/2307.02394/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 5 Jul 2023.

## Builds on (most cited first, 29 of 29)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [Alignment of Language Agents](https://mlsanity.com/paper/2103.14659/) (2021)
- [Hurdles to Progress in Long-form Question Answering](https://mlsanity.com/paper/2103.06332/) (2021)
- [Think you have Solved Direct-Answer Question Answering? Try ARC-DA, the Direct-Answer AI2 Reasoning Challenge](https://mlsanity.com/paper/2102.03315/) (2021)
- [Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models](https://mlsanity.com/paper/2103.06922/) (2021)
- [CREPE: Open-Domain Question Answering with False Presuppositions](https://mlsanity.com/paper/2211.17257/) (2022)
- [GenericsKB: A Knowledge Base of Generic Statements](https://mlsanity.com/paper/2005.00660/) (2020)
- [Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrieval](https://mlsanity.com/paper/2010.11915/) (2020)
- [Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering](https://mlsanity.com/paper/2101.00391/) (2021)
- [General-Purpose Question-Answering with Macaw](https://mlsanity.com/paper/2109.02593/) (2021)
- [Unanswerable Questions about Images and Texts](https://mlsanity.com/paper/2102.06793/) (2021)

## Built on by (most cited first, 30 of 48)

- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://mlsanity.com/paper/2411.15124/) (2024)
- [UltraFeedback: Boosting Language Models with Scaled AI Feedback](https://mlsanity.com/paper/2310.01377/) (2023)
- [Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration](https://mlsanity.com/paper/2402.00367/) (2024)
- [AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions](https://mlsanity.com/paper/2506.09038/) (2025)
- [Preference Leakage: A Contamination Problem in LLM-as-a-judge](https://mlsanity.com/paper/2502.01534/) (2025)
- [R-Tuning: Instructing Large Language Models to Say `I Don't Know'](https://mlsanity.com/paper/2311.09677/) (2023)
- [SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales](https://mlsanity.com/paper/2405.20974/) (2024)
- [InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling](https://mlsanity.com/paper/2402.09345/) (2024)
- [Self-Boosting Large Language Models with Synthetic Preference Data](https://mlsanity.com/paper/2410.06961/) (2024)
- [Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledge](https://mlsanity.com/paper/2311.09731/) (2023)
- [Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluation](https://mlsanity.com/paper/2502.06563/) (2025)
- [A Survey on the Honesty of Large Language Models](https://mlsanity.com/paper/2409.18786/) (2024)
- [How to Protect Yourself from 5G Radiation? Investigating LLM Responses to Implicit Misinformation](https://mlsanity.com/paper/2503.09598/) (2025)
- [Don't Let It Hallucinate: Premise Verification via Retrieval-Augmented Logical Reasoning](https://mlsanity.com/paper/2504.06438/) (2025)
- [Agentic Abstention: Do Agents Know When to Stop Instead of Act?](https://mlsanity.com/paper/2606.28733/) (2026)
- [Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL](https://mlsanity.com/paper/2604.17073/) (2026)
- [MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibration](https://mlsanity.com/paper/2505.23224/) (2025)
- [Over-Searching in Search-Augmented Large Language Models](https://mlsanity.com/paper/2601.05503/) (2026)
- [KG-FPQ: Evaluating Factuality Hallucination in LLMs with Knowledge Graph-based False Premise Questions](https://mlsanity.com/paper/2407.05868/) (2024)
- [MultiHoax: A Dataset of Multi-hop False-Premise Questions](https://mlsanity.com/paper/2506.00264/) (2025)
- [RefusalBench: Generative Evaluation of Selective Refusal in Grounded Language Models](https://mlsanity.com/paper/2510.10390/) (2025)
- [Toward Robust RALMs: Revealing the Impact of Imperfect Retrieval on Retrieval-Augmented Language Models](https://mlsanity.com/paper/2410.15107/) (2024)
- [Well Begun is Half Done: Low-resource Preference Alignment by Weak-to-Strong Decoding](https://mlsanity.com/paper/2506.07434/) (2025)
- [Do What? Teaching Vision-Language-Action Models to Reject the Impossible](https://mlsanity.com/paper/2508.16292/) (2025)
- [I Could've Asked That: Reformulating Unanswerable Questions](https://mlsanity.com/paper/2407.17469/) (2024)
- [SIMPLEMIX: Frustratingly Simple Mixing of Off- and On-policy Data in Language Model Preference Learning](https://mlsanity.com/paper/2505.02363/) (2025)
- [DogeRM: Equipping Reward Models with Domain Knowledge through Model Merging](https://mlsanity.com/paper/2407.01470/) (2024)
- [Information-Theoretic Reward Modeling for Stable RLHF: Detecting and Mitigating Reward Hacking](https://mlsanity.com/paper/2510.13694/) (2025)
- [MAC-Tuning: LLM Multi-Compositional Problem Reasoning with Enhanced Knowledge Boundary Awareness](https://mlsanity.com/paper/2504.21773/) (2025)
- [Reverse Engineering Human Preferences with Reinforcement Learning](https://mlsanity.com/paper/2505.15795/) (2025)

## Abstract (arXiv)

Pre-trained language models (PLMs) have shown unprecedented potential in various fields, especially as the backbones for question-answering (QA) systems. However, they tend to be easily deceived by tricky questions such as "How many eyes does the sun have?". Such frailties of PLMs often allude to the lack of knowledge within them. In this paper, we find that the PLMs already possess the knowledge required to rebut such questions, and the key is how to activate the knowledge. To systematize this observation, we investigate the PLMs' responses to one kind of tricky questions, i.e., the false premises questions (FPQs). We annotate a FalseQA dataset containing 2365 human-written FPQs, with the corresponding explanations for the false premises and the revised true premise questions. Using FalseQA, we discover that PLMs are capable of discriminating FPQs by fine-tuning on moderate numbers (e.g., 256) of examples. PLMs also generate reasonable explanations for the false premise, which serve as rebuttals. Further replaying a few general questions during training allows PLMs to excel on FPQs and general questions simultaneously. Our work suggests that once the rebuttal ability is stimulated, knowledge inside the PLMs can be effectively utilized to handle FPQs, which incentivizes the research on PLM-based QA systems.