mlsanityData to 14 Oct 2026

Predictive Entropy as a Joint Screen for Error and Paraphrase Instability in Medical Vision-Language Models

arXiv 2604.08941 ↗First posted 10 Apr 2026cs.LGcited by 2

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Medical Vision-Language Models (VLMs) answering binary presence questions on chest radiographs can fail in two linked ways: they are confidently wrong, and they change answers when a clinically equivalent question is rephrased. In a binary answer head both failures track the same logit margin, so we test how well one score screens for both. On MedGemma-4B-IT across MIMIC-CXR (in-distribution, n=196) and PadChest (out-of-distribution, n=861), single-pass predictive entropy predicts which yes/no answers flip under rephrasing with an area under the receiver operating characteristic curve (AUROC) of 0.823 on PadChest and 0.821 on MIMIC, and the signal is stable across five random seeds (0.828±0.021). The same signal replicates on LLaVA-RAD-7B (AUROC 0.83) and is invariant to the flip definition (all, operator-preserving, or negation-excluded rewrites), so the screen needs no semantic filter at inference. One low-entropy threshold drives error and flip rate down together (3.3% and 5.7% at 80% coverage), and threshold selection transfers to a held-out split. Under cross-site shift the more expensive methods we test add nothing practical: at the 5% risk target a single forward pass answers 88.5% of cases, matching Monte Carlo (MC) Dropout, and beats the uniformly averaged five-seed adapter ensemble on error detection by 0.111 AUROC. Even a model that is 91% accurate still contradicts itself on 13% of items under rephrasing, and one forward pass flags them.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 15 of its 23 references that are in the map. First posted on arXiv 10 Apr 2026.

Builds on

15 of 23 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Mechanistically Guided LoRA Improves Paraphrase Consistency in Medical Vision-Language Models24 Feb
  2. 2026PSF-Med: Measuring and Explaining Paraphrase Sensitivity in Medical Vision Language Models24 Feb
  3. 2024SURE-VQA: Systematic Understanding of Robustness Evaluation in Medical VQA Tasks29 Nov

Other references · 12

  1. 2017On Calibration of Modern Neural Networkscited by 4,960
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  4. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  5. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  6. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792
  7. 2025MedGemma Technical Reportcited by 436
  8. 2019PadChest: A large chest x-ray image dataset with multi-label annotated reportscited by 309
  9. 2018Understanding Measures of Uncertainty for Adversarial Example Detectioncited by 177
  10. 2023LoRA ensembles for large language model fine-tuningcited by 67
  11. 2023Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shiftcited by 20
  12. 2024LoRA-Ensemble: Efficient Uncertainty Modelling for Self-Attention Networkscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Attention Without Grounding: Causal Evaluation of Visual Explanations in Medical VLMscited by 1
  2. 2026Auditing Medical Vision-Language Models on Chest Radiographs: Estimating Reference Agreement Across Institutionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbations
  2. 2026Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation
  3. 2026Toward Guarantees for Clinical Reasoning in Vision Language Models via Formal Verification
  4. 2026MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence
  5. 2026Consistent but Dangerous: Per-Sample Safety Classification Reveals False Reliability in Medical Vision-Language Models
  6. 2026CXR-ContraBench: Benchmarking Negated-Option Attraction in Medical VLMs
  7. 2026Calibrated Triage, Not Autonomy: Confidence Estimation for Medical Vision-Language Models
  8. 2026To Agree or To Be Right? The Grounding-Sycophancy Tradeoff in Medical Vision-Language Models