mlsanityData to 14 Oct 2026

MedDialBench: Benchmarking LLM Diagnostic Robustness under Parametric Adversarial Patient Behaviors

arXiv 2604.06846 ↗First posted 8 Apr 2026cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Interactive medical dialogue benchmarks have shown that LLM diagnostic accuracy degrades significantly when interacting with non-cooperative patients, yet existing approaches either apply adversarial behaviors without graded severity or case-specific grounding, or reduce patient non-cooperation to a single ungraded axis, and none analyze cross-dimension interactions. We introduce MedDialBench, a benchmark enabling controlled, dose-response characterization of how individual patient behavior dimensions affect LLM diagnostic robustness. It decomposes patient behavior into five dimensions -- Logic Consistency, Health Cognition, Expression Style, Disclosure, and Attitude -- each with graded severity levels and case-specific behavioral scripts. This controlled factorial design enables graded sensitivity analysis, dose-response profiling, and cross-dimension interaction detection. Evaluating five frontier LLMs across 7,225 dialogues (85 cases x 17 configurations x 5 models), we find a fundamental asymmetry: information pollution (fabricating symptoms) produces 1.7-3.4x larger accuracy drops than information deficit (withholding information), and fabricating is the only configuration achieving statistical significance across all five models (McNemar p < 0.05). Among six dimension combinations, fabricating is the sole driver of super-additive interaction: all three fabricating-involving pairs produce O/E ratios of 0.70-0.81 (35-44% of eligible cases fail under the combination despite succeeding under each dimension alone), while all non-fabricating pairs show purely additive effects (O/E ~ 1.0). Inquiry strategy moderates deficit but not pollution: exhaustive questioning recovers withheld information, but cannot compensate for fabricated inputs. Models exhibit distinct vulnerability profiles, with worst-case drops ranging from 38.8 to 54.1 percentage points.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 16 of its 26 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

16 of 26 references.

References · 16

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  3. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  4. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 543
  5. 2024Capabilities of Gemini Models in Medicinecited by 237
  6. 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environmentscited by 231
  7. 2024MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoningcited by 154
  8. 2024MedFuzz: Exploring the Robustness of Large Language Models in Medical Question Answeringcited by 26
  9. 2024Simulated patient systems powered by large language model-based AI agents offer potential for transforming medical educationcited by 13
  10. 2026MedDialogRubrics: A Comprehensive Benchmark and Evaluation Framework for Multi-turn Medical Consultations in Large Language Modelscited by 12
  11. 2025The Dialogue That Heals: A Comprehensive Evaluation of Doctor Agents' Inquiry Capabilitycited by 10
  12. 2025Clinical knowledge in LLMs does not translate to human interactionscited by 7
4 more
  1. 2026SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Carecited by 5
  2. 2026LingxiDiagBench: A Multi-Agent Framework for Benchmarking LLMs in Chinese Psychiatric Consultation and Diagnosiscited by 4
  3. 2026Beyond Idealized Patients: Evaluating LLMs under Challenging Patient Behaviors in Medical Consultationscited by 3
  4. 2025MedPI: Evaluating AI Systems in Medical Patient-facing Interactionscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.