# From Hindsight to Foresight: Dissecting Clinical Evidence Acquisition in Large Language Models

arXiv: https://arxiv.org/abs/2610.18605 · HTML: https://mlsanity.com/paper/2610.18605/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 29 references that are in the map.
- First posted on arXiv 13 Oct 2026.

## Builds on (most cited first, 15 of 15)

- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning](https://mlsanity.com/paper/2406.00922/) (2024)
- [AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator](https://mlsanity.com/paper/2402.09742/) (2024)
- [Sequential Diagnosis with Language Models](https://mlsanity.com/paper/2506.22405/) (2025)
- [DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Models](https://mlsanity.com/paper/2505.14107/) (2025)
- [Doctor-R1: Mastering Clinical Inquiry with Experiential Agentic Reinforcement Learning](https://mlsanity.com/paper/2510.04284/) (2025)
- [Follow-up Question Generation For Enhanced Patient-Provider Conversations](https://mlsanity.com/paper/2503.17509/) (2025)
- [MedClarify: An information-seeking AI agent for medical diagnosis with case-specific follow-up questions](https://mlsanity.com/paper/2602.17308/) (2026)
- [Strong Reasoning Isn't Enough: Evaluating Evidence Elicitation in Interactive Diagnosis](https://mlsanity.com/paper/2601.19773/) (2026)
- [MedAction: Towards Active Multi-turn Clinical Diagnostic LLMs](https://mlsanity.com/paper/2605.07305/) (2026)
- [DDO: Dual-Decision Optimization for LLM-Based Medical Consultation via Multi-Agent Collaboration](https://mlsanity.com/paper/2505.18630/) (2025)
- [Linking Knowledge to Care: Knowledge Graph-Augmented Medical Follow-Up Question Generation](https://mlsanity.com/paper/2603.01252/) (2026)
- [Do LLMs Know What to Ask and When? Evaluating Multi-Turn Information Seeking](https://mlsanity.com/paper/2608.14808/) (2026)
- [Auditing Evidence Use in Medical LLM Diagnosis](https://mlsanity.com/paper/2607.20848/) (2026)
- [Hindsight Bias in Clinical Temporal Reasoning: How Future Data Exposure Affects Large Language Model Judgment](https://mlsanity.com/paper/2609.13454/) (2026)

## Abstract (arXiv)

Large language models (LLMs) often diagnose full sight, static clinical cases well yet struggle in foresight, dynamic diagnosis. Prior work has improved this limitation by quantifying the information models acquire and by developing stronger follow-up strategies to improve diagnostic accuracy. In contrast, we introduce a distinctive hindsight to systematically dissect why LLMs fail to acquire diagnostically critical evidence, tracing the failure from evidence identification, utilization and downstream diagnostic updating through not only diagnostic accuracy, but also evidence coverage, trajectory dynamics, and elicited probability and information entropy. Starting experiments from the breakdown at the level of capabilities, to the diagnosis of the pre-event state before the evidence arrives, and to the post-event state transition after the evidence arrives, our research has revealed the profound limitations and internal causes existing in the dynamic diagnostic process. At the same time, we have also conducted sufficient verification and generalization experiments to prove that these limitations are widespread. Together, these findings shift the central challenge from whether LLMs possess relevant clinical knowledge to whether they can identify, acquire, and effectively use the right evidence under clinical diagnosis and conduct a more detailed mechanism analysis to identify the exact location of the capability gap.