mlsanityData to 14 Oct 2026

Beyond the Leaderboard: Design Lessons for Trustworthy Multimodal VQA

arXiv 2607.15241 ↗First posted 16 Jul 2026cs.CL cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Healthcare multimodal AI must combine visual and textual evidence while remaining reliable and interpretable. Using MediaEval Medico 2025 as a retrospective GI endoscopy case study, we analyze design choices across nine documented systems for question answering and explanation quality. Parameter-efficient adaptation of pretrained backbones provides strong challenge performance, but answer-level gains do not consistently translate into faithful and complete clinical reasoning. Methods enforcing structured reasoning and explicit grounding show more reliable behavior across heterogeneous question types, although the evidence is correlational rather than ablation-based. These results motivate evaluation beyond lexical overlap, standardized evidence-linked explanations, leakage-aware data governance, and lightweight robustness and calibration checks. The findings support trustworthy multimodal healthcare AI based on data fusion, explainability, and resilient evaluation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 16 Jul 2026.

Builds on

26 of 37 references.

References · 26

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  9. 2020Quantifying Attention Flow in Transformerscited by 775
  10. 2024PaliGemma: A versatile 3B VLM for transfercited by 696
  11. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
  12. 2021Image Segmentation Using Text and Image Promptscited by 512
14 more
  1. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
  2. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
  3. 2021Curriculum Learning: A Surveycited by 259
  4. 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 230
  5. 2024PaliGemma 2: A Family of Versatile VLMs for Transfercited by 207
  6. 2018Multimodal Explanations: Justifying Decisions and Pointing to the Evidencecited by 179
  7. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
  8. 2021Medical Visual Question Answering: A Surveycited by 93
  9. 2023LLM-FP4: 4-Bit Floating-Point Quantized Transformerscited by 69
  10. 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
  11. 2024Kvasir-VQA: A Text-Image Pair GI Tract Datasetcited by 23
  12. 2025Kvasir-VQA-x1: A Multimodal Dataset for Medical Reasoning and Robust MedVQA in Gastrointestinal Endoscopycited by 11
  13. 2025Medico 2025: Visual Question Answering for Gastrointestinal Imagingcited by 3
  14. 2025Multi-Task Learning for Visually Grounded Reasoning in Gastrointestinal VQAcited by 2

Built on by

Not cited yet.