mlsanityData to 7 Oct 2026

Assessing VLM Reliability for Medical Image Quality Evaluation Under Corruption and Bias

arXiv 2607.01973 ↗First posted 2 Jul 2026cs.CV cs.AI cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Vision-Language Models (VLMs) are increasingly applied in medical tasks such as pathology description, report generation, and visual question answering. Medical Image Quality Assessment (MIQA) supports diagnostic accuracy and patient safety by determining whether images meet the standards required for clinical decision-making. Automating MIQA with VLMs may reduce workload, but their behavior under real-world conditions, where images may be degraded or textual context may affect judgments, should be further explored before deployment. We benchmark VLMs on medical image quality using the MediMeta-C dataset zero-shot across seven corruption types and five severity levels. We evaluate sensitivity to degradation patterns, the effect of corruptions on embedding geometry, and whether textual attributes (demographics, expertise, infrastructure, institution) alter scores. Across 16 VLMs and seven modalities, pixelation produced the largest score reductions (mean -20.58%, up to -34.4% for OCT), whereas brightness had limited effect (-0.81%). Embedding displacement was associated with score changes. Same-family models showed correlations of 0.67-0.83; some produced increases up to +31% for corrupted mammography. Textual attributes affected scores: institutional prestige raised them +17.15%, and equipment age lowered them -14.7%. The largest changes were +95.62% (InternVL-8B) and -37.7% (MedGemma). Current VLMs show limitations for medical image quality assessment. Pixelation, a privacy-preserving transformation, reduces performance, indicating a trade-off between patient privacy and reliability. Sensitivity to contextual metadata indicates limited objectivity and marks metadata as a privacy and bias source. Privacy protection and objective quality assessment are related requirements for use.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 16 of its 26 references that are in the map. First posted on arXiv 2 Jul 2026.

Builds on

16 of 26 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning8 Jun
  2. 2025On the Robustness of Medical Vision-Language Models: Are they Truly Generalizable?21 May
  3. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review4 Mar
  4. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 12

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  4. 2024A Survey on LLM-as-a-Judgecited by 1,384
  5. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 564
  6. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 354
  7. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directionscited by 66
  8. 2025Interpreting the linear structure of vision-language model embedding spacescited by 32
  9. 2025A Survey on Efficient Vision-Language Modelscited by 19
  10. 2024A comprehensive and easy-to-use multi-domain multi-task medical imaging meta-datasetcited by 12
  11. 2024A General Framework for Producing Interpretable Semantic Text Embeddingscited by 10
  12. 2025MedQ-Bench: Evaluating and Exploring Medical Image Quality Assessment Abilities in MLLMscited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026MedQ-Deg: A Multidimensional Benchmark for Evaluating MLLMs Across Medical Image Quality Degradations
  2. 2026MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence
  3. 2024SURE-VQA: Systematic Understanding of Robustness Evaluation in Medical VQA Tasks
  4. 2026A Controlled Audit of Pretraining Contamination in Public Medical Vision-Language Benchmarks
  5. 2026Solve the Missing First Step: Can VLMs Standardize Raw Heterogeneous Medical Data?
  6. 2025MedVision: Benchmarking Quantitative Medical Image Analysis
  7. 2026MedObvious: Exposing the Medical Moravec's Paradox in VLMs via Clinical Triage
  8. 2025How Far Have Medical Vision-Language Models Come? A Comprehensive Benchmarking Study