# Measuring Epistemic Resilience of LLMs Under Misleading Medical Context

arXiv: https://arxiv.org/abs/2606.12291 · HTML: https://mlsanity.com/paper/2606.12291/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 10 Jun 2026.

## Builds on (most cited first, 22 of 22)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering](https://mlsanity.com/paper/2203.14371/) (2022)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [MedGemma Technical Report](https://mlsanity.com/paper/2507.05201/) (2025)
- [PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models](https://mlsanity.com/paper/2402.07867/) (2024)
- [AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments](https://mlsanity.com/paper/2405.07960/) (2024)
- [MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding](https://mlsanity.com/paper/2501.18362/) (2025)
- [Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Dataset](https://mlsanity.com/paper/2306.03030/) (2023)
- [Benchmarking large language models for biomedical natural language processing applications and recommendations](https://mlsanity.com/paper/2305.16326/) (2023)
- [MedAgentBench: A Realistic Virtual EHR Environment to Benchmark Medical LLM Agents](https://mlsanity.com/paper/2501.14654/) (2025)
- [OpenSeeker: Democratizing Frontier Search Agents by Fully Open-Sourcing Training Data](https://mlsanity.com/paper/2603.15594/) (2026)
- [Persuasion with Large Language Models: A Survey of Empirical Evidence, Study Methodologies, and Ethical Implications](https://mlsanity.com/paper/2411.06837/) (2024)
- [MedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models](https://mlsanity.com/paper/2312.12806/) (2023)
- [A Novel Evaluation Benchmark for Medical LLMs: Illuminating Safety and Effectiveness in Clinical Domains](https://mlsanity.com/paper/2507.23486/) (2025)
- [Evaluating Clinical Competencies of Large Language Models with a General Practice Benchmark](https://mlsanity.com/paper/2503.17599/) (2025)
- [MedRiskEval: Medical Risk Evaluation Benchmark of Language Models, On the Importance of User Perspectives in Healthcare Settings](https://mlsanity.com/paper/2507.07248/) (2025)

## Built on by (most cited first, 3 of 3)

- [Reasoning in Real World Clinical Care: Why Large Language Models Are Not Yet Safe for Autonomous Clinical Decision Support](https://mlsanity.com/paper/2607.28677/) (2026)
- [Training-Free Inference-Time Self-Reflection and Cost-Bounded Early Stopping for Large Language Models](https://mlsanity.com/paper/2608.18884/) (2026)
- [Untangling the Mechanisms of Misleading Context in Medical Question Answering](https://mlsanity.com/paper/2609.02754/) (2026)

## Abstract (arXiv)

Large language models (LLMs) now reach expert-level scores on medical licensing exams, encouraging the assumption that high scores imply safe medical judgment while patients increasingly use them for health advice. We show this assumption is fragile: when misleading context is injected into questions that LLMs originally answer correctly, they abandon the correct answer. We call the ability to maintain correct judgment under adversarial context epistemic resilience, and introduce MedMisBench to measure it. MedMisBench contains 10,932 medical question items and 48,889 misleading context-option pairs spanning medical reasoning, agentic capability, and patient-journey evaluation. Across 11 model configurations, mean accuracy falls from 71.1% on original questions to 38.0% under focused misleading context, with 51.5% attack success. The most damaging injections are formal, rule-like fabrications: authority-framed falsehoods reach 69.5% attack success and exception-poisoning claims reach 64.1%. A 14-member clinical panel from 7 countries identified serious potential harm in 38.2% of reviewed cases. MedMisBench exposes a structural blind spot in LLM evaluation in medical settings: existing benchmarks measure what models know, but not whether they preserve correct medical judgment under misleading context.