Unknown is not normal: separating language-model extraction from rule-based decision logic for clinical risk scores
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are increasingly used to compute clinical risk scores from free-text notes. Notes are often incomplete, and treating undocumented findings as normal can silently misclassify patients. We test whether separating three-state extraction (present, absent or unknown, by an LLM) from decision logic (deterministic code computing score bounds over unknown inputs) lets a system ask only questions that can change the decision. On 1,200 synthetic emergency cases across six calculators (HEART, CURB-65, qSOFA, PERC, Wells, Cockcroft-Gault), with a simulated clinician answering questions, we compared this bounds policy with asking for every missing input, a missing-equals-normal schema, and an end-to-end LLM agent (Claude Opus 5.5). With Claude Haiku 4.5 as extractor, the bounds policy matched ask-all accuracy (99.4% vs 99.4%) with half the questions (0.92 vs 1.78 per case) and no irrelevant ones. Treating missing as normal dropped accuracy to 91.2% and under-triaged 8.5% of patients (95% CI 7.1-10.2), and under-triage persisted under messy notes and a noisy clinician. The agent was equally accurate under ideal conditions (99.6%) but 9.5% of its questions were irrelevant; with a noisy clinician it was less accurate than the bounds policy (83.5% vs 87.0%, p<0.001) and committed prematurely in 2.7% of cases (bounds: 0%). A 9B local model as extractor reached oracle-level accuracy (99.8%). In 584 real case reports from MedCalc-Bench, only 52% contained enough information to determine the category (HEART 13%). Routing decisions through code that reasons explicitly about unknowns avoids premature commitment and irrelevant questions, halves the questions asked, and works with small local models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 30 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
10 of 30 references.
References · 10
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2017Selective Classification for Deep Neural Networkscited by 852
- 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environmentscited by 231
- 2024MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoningcited by 154
- 2024MedCalc-Bench: Evaluating Large Language Models for Medical Calculationscited by 70
- 2025From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculationscited by 11
- 2026MedMCP-Calc: Benchmarking LLMs for Realistic Medical Calculator Scenarios via MCP Integrationcited by 3
- 2026ClinDet-Bench: Beyond Abstention, Evaluating Judgment Determinability of LLMs in Clinical Decision-Makingcited by 3
- 2026MedCalc-Bench Doesn't Measure What You Think: A Benchmark Audit and the Case for Open-Book Evaluationcited by 3
- 2025Scalable Stewardship of an LLM-Assisted Clinical Benchmark with Physician Oversightcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026How Robust Are Large Language Models for Clinical Numeracy? An Empirical Study on Numerical Reasoning Abilities in Clinical Contexts
- 2026CLEAR: Revealing How Noise and Ambiguity Degrade Reliability in LLMs for Medicine
- 2026Safety and accuracy follow different scaling laws in clinical large language models
- 2026LLM-as-a-Judge in Healthcare: A Scoping Analysis of Applications, Methods, and Human Alignment
- 2026Measuring Epistemic Resilience of LLMs Under Misleading Medical Context
- 2026Knowing When to Abstain: Medical LLMs Under Clinical Uncertainty
- 2026Medical Reasoning with Large Language Models: A Survey and MR-Bench
- 2025RiskAgent: Synergizing Language Models with Validated Tools for Evidence-Based Risk Prediction