mlsanityData to 14 Oct 2026

Multimodal Retrieval-Augmented Generation with Large Language Models for Medical VQA

arXiv 2510.13856 ↗First posted 12 Oct 2025cs.CL cs.AI cs.CVcited by 3

On the map: Retrieval-augmented generation → Medical and biomedical RAG

Abstract (arXiv)

Medical Visual Question Answering (MedVQA) enables natural language queries over medical images to support clinical decision-making and patient care. The MEDIQA-WV 2025 shared task addressed wound-care VQA, requiring systems to generate free-text responses and structured wound attributes from images and patient queries. We present the MasonNLP system, which employs a general-domain, instruction-tuned large language model with a retrieval-augmented generation (RAG) framework that incorporates textual and visual examples from in-domain data. This approach grounds outputs in clinically relevant exemplars, improving reasoning, schema adherence, and response quality across dBLEU, ROUGE, BERTScore, and LLM-based metrics. Our best-performing system ranked 3rd among 19 teams and 51 submissions with an average score of 41.37%, demonstrating that lightweight RAG with general-purpose LLMs -- a minimal inference-time layer that adds a few relevant exemplars via simple indexing and fusion, with no extra training or complex re-ranking -- provides a simple and effective baseline for multimodal clinical NLP tasks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation. Builds on 20 of its 39 references that are in the map. First posted on arXiv 12 Oct 2025.

Builds on

20 of 39 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Benchmarking Retrieval-Augmented Generation for Medicine20 Feb
  2. 2023RAMM: Retrieval-augmented Biomedical Visual Question Answering with Multi-modal Pre-training1 Mar

Other references · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2015VQA: Visual Question Answeringcited by 2,506
  5. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  6. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  7. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  8. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  9. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 570
  10. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
  11. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
  12. 2019Deep Modular Co-Attention Networks for Visual Question Answeringcited by 339
6 more
  1. 2021Medical Visual Question Answering: A Surveycited by 93
  2. 2021MMBERT: Multimodal BERT Pretraining for Improved Medical VQAcited by 68
  3. 2015Ask Your Neurons: A Neural-based Approach to Answering Questions about Imagescited by 51
  4. 2025MedFrameQA: A Multi-Image Medical VQA Benchmark for Clinical Reasoningcited by 34
  5. 2015deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targetscited by 21
  6. 2024R-LLaVA: Improving Med-VQA Understanding through Visual Region of Interestcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Iterative Multimodal Retrieval-Augmented Generation for Medical Question Answering
  2. 2024Fusion of Domain-Adapted Vision and Language Models for Medical Visual Question Answering
  3. 2024MedThink: Explaining Medical Visual Question Answering via Multimodal Decision-Making Rationale
  4. 2024MISS: A Generative Pretraining and Finetuning Approach for Med-VQA
  5. 2025Synthesizing High-Quality Visual Question Answering from Medical Documents with Generator-Verifier LMMs
  6. 2025ClinKD: Cross-Modal Clinical Knowledge Distiller For Multi-Task Medical Images
  7. 2025A Lightweight Large Vision-language Model for Multimodal Medical Images
  8. 2025Optimizing Medical Question-Answering Systems: A Comparative Study of Fine-Tuned and Zero-Shot Large Language Models with RAG Framework