mlsanityData to 7 Oct 2026

Iterative Multimodal Retrieval-Augmented Generation for Medical Question Answering

arXiv 2604.27724 ↗First posted 30 Apr 2026cs.AIcited by 0

On the map: Retrieval-augmented generation → Medical and biomedical RAG

Abstract (arXiv)

Medical retrieval-augmented generation (RAG) systems typically operate on text chunks extracted from biomedical literature, discarding the rich visual content (tables, figures, structured layouts) of original document pages. We propose MED-VRAG, an iterative multimodal RAG framework that retrieves and reasons over PMC document page images instead of OCR'd text. The system pairs ColQwen2.5 patch-level page embeddings with a sharded MapReduce LLM filter, scaling to ~350K pages while keeping Stage-1 retrieval under 30 ms via an offline coarse-to-fine index (C=8 centroids per page, ANN over centroids, exact two-way scoring on the top-R shortlist). A vision-language model (VLM) then iteratively refines its query and accumulates evidence in a memory bank across up to 3 reasoning rounds, with a single iteration costing ~15.9 s and the full three-round pipeline ~47.8 s on 4xA100. Across four medical QA benchmarks (MedQA, MedMCQA, PubMedQA, MMLU-Med), MEDVRAG reaches 78.6% average accuracy. Under controlled comparison with the same Qwen2.5-VL-32B backbone, retrieval contributes a +5.8 point gain over the no-retrieval baseline; we also note a +1.8 point edge over MedRAG + GPT-4 (76.8%), with the caveat that this is a cross-paper rather than head-to-head comparison. Ablations isolate +1.0 from page-image vs text-chunk retrieval, +1.5 from iteration, and +1.0 from the memory bank.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation. Builds on 28 of its 32 references that are in the map. First posted on arXiv 30 Apr 2026.

Builds on

28 of 32 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Rationale-Guided Retrieval Augmented Generation for Medical Question Answering1 Nov
  2. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Models16 Oct
  3. 2024Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questions1 Aug
  4. 2024Benchmarking Retrieval-Augmented Generation for Medicine20 Feb
  5. 2024Improving Medical Reasoning through Retrieval and Self-Reflection with Retrieval-Augmented Large Language Models27 Jan

Other references · 23

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  6. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  8. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  9. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  10. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  11. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  12. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
11 more
  1. 2023Active Retrieval Augmented Generationcited by 506
  2. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 423
  3. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  4. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 272
  5. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
  6. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 237
  7. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understandingcited by 110
  8. 2024Unifying Multimodal Retrieval via Document Screenshot Embeddingcited by 107
  9. 2024RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Modelscited by 58
  10. 2025VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documentscited by 57
  11. 2024OpenMedLM: Prompt engineering can out-perform fine-tuning in medical question-answering with open-source large language modelscited by 20

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Multimodal Retrieval-Augmented Generation with Large Language Models for Medical VQA
  2. 2025Optimizing Medical Question-Answering Systems: A Comparative Study of Fine-Tuned and Zero-Shot Large Language Models with RAG Framework
  3. 2026A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering
  4. 2025MIRA: A Novel Framework for Fusing Modalities in Medical RAG
  5. 2025RAG-BioQA: A Retrieval-Augmented Generation Framework for Long-Form Biomedical Question Answering
  6. 2025Efficient and Reproducible Biomedical Question Answering using Retrieval Augmented Generation
  7. 2024Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answering
  8. 2025Synthesizing High-Quality Visual Question Answering from Medical Documents with Generator-Verifier LMMs