mlsanityData to 14 Oct 2026

RoiMAM: Region-of-Interest Medical Attention Model for Efficient Vision-Language Understanding

arXiv 2605.15561 ↗First posted 15 May 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Vision-Language Models (VLMs) facilitate medical visual question answering (MedVQA) by jointly interpreting images and text. However, existing models typically depend on large architectures and closed-set answers, which limits their efficiency and potential clinical applicability. To overcome these shortcomings, we introduce RoiMAM, an efficient VLM. It integrates a training-free ROI Generation Module with Semantic Selective Suppression to focus on lesion-relevant regions, alongside a Text Prompt Enhancer module that provides modality-specific context without introducing training parameters. Compared to the widely used MedVInT-TD model, our design achieves efficient and accurate diagnosis at less than 20% of the model size, while improving accuracy by approximately 2% on SLAKE and 4.6% on PMC-VQA.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 9 of its 16 references that are in the map. First posted on arXiv 15 May 2026.

Builds on

9 of 16 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024UniMed-CLIP: Towards a Unified Image-Text Pretraining Paradigm for Diverse Medical Imaging Modalities13 Dec
  2. 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale27 Jun
  3. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 6

  1. 2024Qwen2 Technical Reportcited by 2,120
  2. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
  3. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
  4. 2023PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documentscited by 212
  5. 2023SAM-Med2Dcited by 124
  6. 2023Visual Explanations of Image-Text Representations via Multi-Modal Information Bottleneck Attributioncited by 34

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Fusion of Domain-Adapted Vision and Language Models for Medical Visual Question Answering
  2. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM
  3. 2026Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMs
  4. 2026MedProb: Probing Internal Representations of Vision-Language Models for Medical Question Answering
  5. 2026EyeVQA: Benchmarking Ophthalmic Vision-Language Models from Recognition to Spatial Grounding
  6. 2025Medical Large Vision Language Models with Multi-Image Visual Ability
  7. 2024Beyond the Hype: A dispassionate look at vision-language models in medical scenario
  8. 2024Targeted Visual Prompting for Medical Visual Question Answering