# Benchmarking Multi-turn Medical Diagnosis: Hold, Lure, and Self-Correction

arXiv: https://arxiv.org/abs/2604.04325 · HTML: https://mlsanity.com/paper/2604.04325/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 6 Apr 2026.

## Builds on (most cited first, 25 of 25)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering](https://mlsanity.com/paper/2203.14371/) (2022)
- [MedGemma Technical Report](https://mlsanity.com/paper/2507.05201/) (2025)
- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs](https://mlsanity.com/paper/2501.17399/) (2025)
- [MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning](https://mlsanity.com/paper/2406.00922/) (2024)
- [Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questions](https://mlsanity.com/paper/2402.18060/) (2024)
- [AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator](https://mlsanity.com/paper/2402.09742/) (2024)
- [Med42-v2: A Suite of Clinical LLMs](https://mlsanity.com/paper/2408.06142/) (2024)
- [Automatic Interactive Evaluation for Large Language Models with State Aware Patient Simulator](https://mlsanity.com/paper/2403.08495/) (2024)
- [StructFlowBench: A Structured Flow Benchmark for Multi-turn Instruction Following](https://mlsanity.com/paper/2502.14494/) (2025)
- [A Novel Evaluation Benchmark for Medical LLMs: Illuminating Safety and Effectiveness in Clinical Domains](https://mlsanity.com/paper/2507.23486/) (2025)
- [Large language models provide unsafe answers to patient-posed medical questions](https://mlsanity.com/paper/2507.18905/) (2025)
- [Parrot: Enhancing Multi-Turn Instruction Following for Large Language Models](https://mlsanity.com/paper/2310.07301/) (2023)
- [When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification](https://mlsanity.com/paper/2602.11199/) (2026)
- [Knowing When to Abstain: Medical LLMs Under Clinical Uncertainty](https://mlsanity.com/paper/2601.12471/) (2026)
- [DiReCT: Diagnostic Reasoning for Clinical Notes via Large Language Models](https://mlsanity.com/paper/2408.01933/) (2024)
- [ClarifyMT-Bench: Benchmarking and Improving Multi-Turn Clarification for Conversational Large Language Models](https://mlsanity.com/paper/2512.21120/) (2025)
- [Follow-up Question Generation For Enhanced Patient-Provider Conversations](https://mlsanity.com/paper/2503.17509/) (2025)
- [Evaluating the Sensitivity of LLMs to Prior Context](https://mlsanity.com/paper/2506.00069/) (2025)
- [Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation](https://mlsanity.com/paper/2602.07338/) (2026)
- [KnowGuard: Knowledge-Driven Abstention for Multi-Round Clinical Reasoning](https://mlsanity.com/paper/2509.24816/) (2025)
- [ClinDet-Bench: Beyond Abstention, Evaluating Judgment Determinability of LLMs in Clinical Decision-Making](https://mlsanity.com/paper/2602.22771/) (2026)
- [TurnWise: The Gap between Single- and Multi-turn Language Model Capabilities](https://mlsanity.com/paper/2603.16759/) (2026)

## Built on by (most cited first, 4 of 4)

- [CDEG: Learning Decision-Critical Evidence for Long-Horizon Diagnostic Agents](https://mlsanity.com/paper/2608.22899/) (2026)
- [Why Didn't It Check? Unsupported Final Claims and Their Repair in Two Tool-Equipped Language Models](https://mlsanity.com/paper/2608.27768/) (2026)
- [Constructing and Evaluating Clinical Reasoning Trajectories for Medical Agent](https://mlsanity.com/paper/2609.05090/) (2026)
- [Can LLMs Value the Right Evidence? Evidence-Value Misalignment in Dynamic Medical Diagnosis](https://mlsanity.com/paper/2609.35627/) (2026)

## Abstract (arXiv)

Large language models (LLMs) achieve high accuracy in medical diagnosis when all clinical information is provided in a single turn, yet how they behave under multi-turn evidence accumulation closer to real clinical reasoning remains unexplored. We introduce MINT (Medical Incremental N-Turn Benchmark), a high-fidelity, multi-turn medical diagnosis benchmark comprising 1,035 cases with clinically labeled evidence shards, controlled turn granularity, and information-preserving decomposition. Through systematic evaluation of 11 LLMs on MINT, we uncover three persistent behavioral patterns that significantly impact diagnostic decisions: (1) intent to answer, models rush to answer before sufficient evidence has been observed, with over 55% of answers committed within the first two turns; (2) self-correction, incorrect-to-correct answer revisions occur at up to 10.6 times the rate of correct-to-incorrect flips, revealing a latent capacity for self-correction that premature commitment forecloses; and (3) strong lures, clinically salient information such as laboratory results trigger premature answering even when models are explicitly instructed to wait. We translate these findings into clinically actionable guidance: deferring the diagnostic question to later turns reduces premature answering and improves accuracy at the first point of commitment by up to 62.6%, while reserving salient clinical evidence for later turns prevents a catastrophic accuracy drop of up to 23.3% caused by premature commitment. Our work provides both a controlled evaluation framework and concrete recommendations for improving the reliability of LLMs in multi-turn medical diagnosis.