mlsanityData to 7 Oct 2026

Evaluating Reasoning Models for Queries with Presuppositions

arXiv 2605.03050 ↗First posted 4 May 2026cs.CLcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Millions of users turn to AI models for their information needs. It is conceivable that a large number of user queries contain assumptions that may be factually inaccurate. Prior work notes that large language models (LLMs) often fail to challenge such erroneous assumptions, and can reinforce users' misinformed opinions. However, given the recent advances, especially in model's reasoning capabilities, we revisit whether large reasoning models (LRMs) can reason about the underlying assumptions and respond to user queries appropriately. We construct queries with varying degrees of presuppositions spanning health, science, and general knowledge, and use it to evaluate several widely-deployed models When compared to non-reasoning models, we find that reasoning models achieve a slightly higher accuracy (2-11%), but they still fail to challenge a large fraction (26-42%) of false presuppositions. Further, reasoning models remain susceptible to how strongly the presupposition is expressed.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 13 of its 27 references that are in the map. First posted on arXiv 4 May 2026.

Builds on

13 of 27 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Qwen3 Technical Report14 May
  2. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models16 Jan

Other references · 11

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  3. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  4. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  5. 2025ELEPHANT: Measuring and understanding social sycophancy in LLMscited by 115
  6. 2025Are Reasoning Models More Prone to Hallucination?cited by 62
  7. 2025Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Modelscited by 18
  8. 2025Cancer-Myth: Evaluating Large Language Models on Patient Questions with False Presuppositionscited by 15
  9. 2025LLMs Struggle to Reject False Presuppositions when Misinformation Stakes are Highcited by 11
  10. 2025Reasoning Isn't Enough: Examining Truth-Bias and Sycophancy in LLMscited by 11
  11. 2024LLM Robustness Against Misinformation in Biomedical Question Answeringcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond
  2. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
  3. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
  4. 2024Critical-Questions-of-Thought: Steering LLM reasoning with Argumentative Querying
  5. 2022Towards Reasoning in Large Language Models: A Survey
  6. 2025Towards Evaluting Fake Reasoning Bias in Language Models
  7. 2025What Defines Good Reasoning in LLMs? Dissecting Reasoning Steps with Multi-Aspect Evaluation
  8. 2026Evaluating the False Trust Engendered by LLM Explanations