# Cancer-Myth: Evaluating Large Language Models on Patient Questions with False Presuppositions

arXiv: https://arxiv.org/abs/2504.11373 · HTML: https://mlsanity.com/paper/2504.11373/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 55 references that are in the map.
- First posted on arXiv 15 Apr 2025.

## Builds on (most cited first, 30 of 36)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering](https://mlsanity.com/paper/2203.14371/) (2022)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [PathVQA: 30000+ Questions for Medical Visual Question Answering](https://mlsanity.com/paper/2003.10286/) (2020)
- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering](https://mlsanity.com/paper/2305.10415/) (2023)
- [FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation](https://mlsanity.com/paper/2310.03214/) (2023)
- [MEDITRON-70B: Scaling Medical Pretraining for Large Language Models](https://mlsanity.com/paper/2311.16079/) (2023)
- [MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making](https://mlsanity.com/paper/2404.15155/) (2024)
- [Adversarial Filters of Dataset Biases](https://mlsanity.com/paper/2002.04108/) (2020)
- [MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning](https://mlsanity.com/paper/2406.00922/) (2024)
- [Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questions](https://mlsanity.com/paper/2402.18060/) (2024)
- [Sycophancy in Large Language Models: Causes and Mitigations](https://mlsanity.com/paper/2411.15287/) (2024)
- [Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation](https://mlsanity.com/paper/2104.08678/) (2021)
- [CREPE: Open-Domain Question Answering with False Presuppositions](https://mlsanity.com/paper/2211.17257/) (2022)
- [The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness](https://mlsanity.com/paper/2401.00287/) (2023)
- [(QA)2: Question Answering with Questionable Assumptions](https://mlsanity.com/paper/2212.10003/) (2022)
- [Augmented Large Language Models with Parametric Knowledge Guiding](https://mlsanity.com/paper/2305.04757/) (2023)
- [A Dataset for Medical Instructional Video Classification and Question Answering](https://mlsanity.com/paper/2201.12888/) (2022)
- [TLDR: Token-Level Detective Reward Model for Large Vision Language Models](https://mlsanity.com/paper/2410.04734/) (2024)
- [Pregnant Questions: The Importance of Pragmatic Awareness in Maternal Health Question Answering](https://mlsanity.com/paper/2311.09542/) (2023)

## Built on by (most cited first, 15 of 15)

- [How RLHF Amplifies Sycophancy](https://mlsanity.com/paper/2602.01002/) (2026)
- [Verbalizing LLMs' assumptions to explain and control sycophancy](https://mlsanity.com/paper/2604.03058/) (2026)
- [Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents](https://mlsanity.com/paper/2510.17491/) (2025)
- [Language Models Encode the Contextual Truth of Propositions](https://mlsanity.com/paper/2608.03035/) (2026)
- [Large Language Models for Cancer Communication: Evaluating Linguistic Quality, Safety, and Accessibility in Generative AI](https://mlsanity.com/paper/2505.10472/) (2025)
- [Evaluating Reasoning Models for Queries with Presuppositions](https://mlsanity.com/paper/2605.03050/) (2026)
- [EUDAIMONIA: Evaluating Undesirable Dynamics in AI](https://mlsanity.com/paper/2605.30654/) (2026)
- [Don't `Well, Actually' Me Unless You Know What You're Talking About: Weak Presupposition Verification Degrades General QA Performance](https://mlsanity.com/paper/2608.06539/) (2026)
- [Safer in Translation? Presupposition Robustness in Indic Languages](https://mlsanity.com/paper/2511.01360/) (2025)
- [MedRedFlag: Investigating how LLMs Redirect Misconceptions in Real-World Health Communication](https://mlsanity.com/paper/2601.09853/) (2026)
- [GraphFlow: An Architecture for Formally Verifiable Visual Workflows Enabling Reliable Agentic AI Automation](https://mlsanity.com/paper/2605.14968/) (2026)
- [What's in a Name? Morphological Shortcuts by LLMs in Pharmacology](https://mlsanity.com/paper/2606.05616/) (2026)
- [Evaluating Large Language Models on Misconceptions in Multi-Turn Medical Conversations](https://mlsanity.com/paper/2607.12884/) (2026)
- [Untangling the Mechanisms of Misleading Context in Medical Question Answering](https://mlsanity.com/paper/2609.02754/) (2026)
- [XYEval: Agents say yes to bad advice](https://mlsanity.com/paper/2609.23939/) (2026)

## Abstract (arXiv)

Cancer patients are increasingly turning to large language models (LLMs) for medical information, making it critical to assess how well these models handle complex, personalized questions. However, current medical benchmarks focus on medical exams or consumer-searched questions and do not evaluate LLMs on real patient questions with patient details. In this paper, we first have three hematology-oncology physicians evaluate cancer-related questions drawn from real patients. While LLM responses are generally accurate, the models frequently fail to recognize or address false presuppositions in the questions, posing risks to safe medical decision-making. To study this limitation systematically, we introduce Cancer-Myth, an expert-verified adversarial dataset of 585 cancer-related questions with false presuppositions. On this benchmark, no frontier LLM -- including GPT-5, Gemini-2.5-Pro, and Claude-4-Sonnet -- corrects these false presuppositions more than 43% of the time. To study mitigation strategies, we further construct a 150-question Cancer-Myth-NFP set, in which physicians confirm the absence of false presuppositions. We find typical mitigation strategies, such as adding precautionary prompts with GEPA optimization, can raise accuracy on Cancer-Myth to 80%, but at the cost of misidentifying presuppositions in 41% of Cancer-Myth-NFP questions and causing a 10% relative performance drop on other medical benchmarks. These findings highlight a critical gap in the reliability of LLMs, show that prompting alone is not a reliable remedy for false presuppositions, and underscore the need for more robust safeguards in medical AI systems.