Large language models exhibit unreliable updating of clinical judgment as patient evidence evolves
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are increasingly explored for clinical reasoning, but whether they appropriately revise judgments as patient evidence evolves remains unclear. We evaluated longitudinal belief updating using matched intensive-care trajectories from electronic health records. Across diverse LLMs, conditioning on a preceding judgment more often increased than reduced prediction error when estimates changed, replicated for a second endpoint. Controlled interventions revealed two failure modes. First, with preceding assessment fixed, models responded more strongly to worsening than matched improving respiratory evidence; this asymmetry persisted after headroom normalization at moderate and strong evidence levels. Second, with current evidence fixed, increasing prior risk from 10% to 90% shifted estimates by 26.2 percentage points, demonstrating causal influence of prior model beliefs. Prompting did not restore reliable updating. Evidence-Validated Longitudinal Update (EVLU) identified fewer, more reliable revisions, revealing a reliability-coverage trade-off. These findings establish longitudinal belief updating as a distinct dimension of LLM reliability.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 27 references that are in the map. First posted on arXiv 2 Oct 2026.
Builds on
11 of 27 references.
References · 11
- 2022Large Language Models Encode Clinical Knowledgecited by 1,830
- 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
- 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 246
- 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
- 2024MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Modelscited by 81
- 2023MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Recordscited by 56
- 2024Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?cited by 48
- 2025TIMER: Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Recordscited by 34
- 2024Belief Revision: The Adaptability of Large Language Models Reasoningcited by 30
- 2025Large Language Models with Temporal Reasoning for Longitudinal Clinical Summarization and Predictioncited by 12
- 2025CliCARE: Grounding Large Language Models in Clinical Guidelines for Decision Support over Longitudinal Cancer Electronic Health Recordscited by 4
Built on by
Not cited yet.