mlsanityData to 14 Oct 2026

The Illusion of Readiness in Health AI

arXiv 2509.18234 ↗First posted 22 Sep 2025cs.AI cs.CL cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Large language models have demonstrated remarkable performance in a wide range of medical benchmarks. Yet underneath the seemingly promising results lie salient growth areas, especially in cutting-edge frontiers such as multimodal reasoning. In this paper, we introduce a series of adversarial stress tests to systematically assess the robustness of flagship models and medical benchmarks. Our study reveals prevalent brittleness in the presence of simple adversarial transformations: leading systems can guess the right answer even with key inputs removed, yet may get confused by the slightest prompt alterations, while fabricating convincing yet flawed reasoning traces. Using clinician-guided rubrics, we demonstrate that popular medical benchmarks vary widely in what they truly measure. Our study reveals significant competency gaps of frontier AI in attaining real-world readiness for health applications. If we want AI to earn trust in healthcare, we must demand more than leaderboard wins and must hold AI systems accountable to ensure robustness, sound reasoning, and alignment with real medical demands.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 18 of its 27 references that are in the map. First posted on arXiv 22 Sep 2025.

Builds on

18 of 27 references.

References · 18

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2013Intriguing properties of neural networkscited by 5,155
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  6. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  7. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  8. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  9. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
  10. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
  11. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
  12. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
6 more
  1. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 329
  2. 2024Capabilities of Gemini Models in Medicinecited by 237
  3. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 184
  4. 2024Towards Conversational Diagnostic AIcited by 79
  5. 2025Sequential Diagnosis with Language Modelscited by 60
  6. 2024Hidden flaws behind expert-level accuracy of multimodal GPT-4 vision in medicinecited by 31

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 8.

12 papers.

  1. 2026Knowledge Graphs are Implicit Reward Models: Path-Derived Signals Enable Compositional Reasoningcited by 7
  2. 2025Agentic Systems in Radiology: Design, Applications, Evaluation, and Challengescited by 5
  3. 2025Fusion or Confusion? Multimodal Complexity Is Not All You Needcited by 5
  4. 2025Multi-modal Data Spectrum: Multi-modal Datasets are Multi-dimensionalcited by 3
  5. 2025Are Large Vision Language Models Truly Grounded in Medical Images? Evidence from Italian Clinical Visual Question Answeringcited by 3
  6. 2026The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don'tcited by 3
  7. 2026Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Taskscited by 2
  8. 2026The Reasoning Trap: An Information-Theoretic Bound on Closed-System Multi-Step LLM Reasoningcited by 2
  9. 2026OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Modelscited by 1
  10. 2026Leveraging Computerized Adaptive Testing for Cost-effective Evaluation of Large Language Models in Medical Benchmarkingcited by 0
  11. 2026Using profiles of cognitive capability to assess AI suitability for workplace taskscited by 0
  12. 2026Repair Before Reinforce: Context-Augmented Knowledge Graph Reasoning for Multi-Hop Question Answeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Addressing Benchmarking Gaps in Large Language Models for Health and Medicine with Dynamic Red-Teaming
  2. 2026Med-R2: An Adversarial Benchmark for Evidence-Grounded Reasoning in Medical VLMs
  3. 2026When Large Language Models Fail in Healthcare: Evaluating Sensitivity to Prompt Variations
  4. 2026Measuring Epistemic Resilience of LLMs Under Misleading Medical Context
  5. 2024Ensuring Safety and Trust: Analyzing the Risks of Large Language Models in Medicine
  6. 2025SafeMed-R1: Adversarial Reinforcement Learning for Generalizable and Robust Medical Reasoning in Vision-Language Models
  7. 2025CARES: Comprehensive Evaluation of Safety and Adversarial Robustness in Medical LLMs
  8. 2026Advances in LLM Reasoning Enable Flexibility in Clinical Problem-Solving