mlsanityData to 14 Oct 2026

Quantifying and Mitigating Premature Closure in Frontier LLMs

arXiv 2605.15000 ↗First posted 14 May 2026cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Premature closure, or committing to a conclusion before sufficient information is available, is a recognized contributor to diagnostic error but remains underexamined in large language models (LLMs). We define LLM premature closure as inappropriate commitment under uncertainty: providing an answer, recommendation, or clinical guidance when the safer response would be clarification, abstention, escalation, or refusal. We evaluated five frontier LLMs across structured and open-ended medical tasks. In MedQA (n = 500) and AfriMed-QA (n = 490) questions where the correct choice had been removed, models still selected an answer at high rates, with baseline false-action rates of 55-81% and 53-82%, respectively. In open-ended evaluation, models gave inappropriate answers on an average of 30% of 861 HealthBench questions and 78% of 191 physician-authored adversarial queries. Safety-oriented prompting reduced premature closure across models, but residual failure persisted, highlighting the need to evaluate whether medical LLMs know when not to answer.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 17 of its 36 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

17 of 36 references.

References · 17

  1. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  2. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  3. 2025Humanity's Last Examcited by 491
  4. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
  5. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 180
  6. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
  7. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 105
  8. 2023Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicinecited by 83
  9. 2024Superhuman performance of a large language model on the reasoning tasks of a physiciancited by 28
  10. 2024Evaluating language models as risk scorescited by 27
  11. 2024AfriMed-QA: A Pan-African, Multi-Specialty, Medical Question-Answering Benchmark Datasetcited by 26
  12. 2026HealthBench Professional: Evaluating Large Language Models on Real Clinician Chatscited by 26
5 more
  1. 2025Large language models provide unsafe answers to patient-posed medical questionscited by 18
  2. 2024Prompt engineering paradigms for medical applications: scoping review and recommendations for better practicescited by 14
  3. 2025Mitigating Over-Refusal in Aligned Large Language Models via Inference-Time Activation Energycited by 5
  4. 2026AI Alignment Breaks at the Edgecited by 4
  5. 2025Prompt engineering does not universally improve Large Language Model performance across clinical decision-making taskscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026InsufficiencyBench: Evaluating LLM legal advice on underspecified user queriescited by 1
  2. 2026Why Didn't It Check? Unsupported Final Claims and Their Repair in Two Tool-Equipped Language Modelscited by 0
  3. 2026Can LLMs Value the Right Evidence? Evidence-Value Misalignment in Dynamic Medical Diagnosiscited by 0
  4. 2026Not Until the Evidence Says So: Teaching LLM Investigators When to Close a Casecited by 0