mlsanityData to 14 Oct 2026

Exploration of Augmentation Strategies in Multi-modal Retrieval-Augmented Generation for the Biomedical Domain: A Case Study Evaluating Question Answering in Glycobiology

arXiv 2512.16802 ↗First posted 18 Dec 2025cs.CLcited by 0

On the map: Retrieval-augmented generation → Medical and biomedical RAG

Abstract (arXiv)

Multi-modal retrieval-augmented generation (MM-RAG) promises grounded biomedical QA, but it is unclear when to (i) convert figures/tables into text versus (ii) use optical character recognition (OCR)-free visual retrieval that returns page images and leaves interpretation to the generator. We study this trade-off in glycobiology, a visually dense domain. We built a benchmark of 120 multiple-choice questions (MCQs) from 25 papers, stratified by retrieval difficulty (easy text, medium figures/tables, hard cross-evidence). We implemented four augmentations-None, Text RAG, Multi-modal conversion, and late-interaction visual retrieval (ColPali)-using Docling parsing and Qdrant indexing. We evaluated mid-size open-source and frontier proprietary models (e.g., Gemma-3-27B-IT, GPT-4o family). Additional testing used the GPT-5 family and multiple visual retrievers (ColPali/ColQwen/ColFlor). Accuracy with Agresti-Coull 95% confidence intervals (CIs) was computed over 5 runs per configuration. With Gemma-3-27B-IT, Text and Multi-modal augmentation outperformed OCR-free retrieval (0.722-0.740 vs. 0.510 average accuracy). With GPT-4o, Multi-modal achieved 0.808, with Text 0.782 and ColPali 0.745 close behind; within-model differences were small. In follow-on experiments with the GPT-5 family, the best results with ColPali and ColFlor improved by ~2% to 0.828 in both cases. In general, across the GPT-5 family, ColPali, ColQwen, and ColFlor were statistically indistinguishable. GPT-5-nano trailed larger GPT-5 variants by roughly 8-10%. Pipeline choice is capacity-dependent: converting visuals to text lowers the reader burden and is more reliable for mid-size models, whereas OCR-free visual retrieval becomes competitive under frontier models. Among retrievers, ColFlor offers parity with heavier options at a smaller footprint, making it an efficient default when strong generators are available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation. Builds on 19 of its 39 references that are in the map. First posted on arXiv 18 Dec 2025.

Builds on

19 of 39 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Biomedical Literature Q&A System Using Retrieval-Augmented Generation (RAG)5 Sep

Other references · 18

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  5. 2025Gemma 3 Technical Reportcited by 1,824
  6. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  7. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  8. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  9. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  10. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 578
  11. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 392
  12. 2023Towards Generalist Biomedical AIcited by 289
6 more
  1. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 245
  2. 2024ColPali: Efficient Document Retrieval with Vision Language Modelscited by 195
  3. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 178
  4. 2025ViDoRe Benchmark V2: Raising the Bar for Visual Retrievalcited by 61
  5. 2024MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Modelscited by 42
  6. 2024Docling Technical Reportcited by 34

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Benchmarking Retrieval Strategies for Biomedical Retrieval-Augmented Generation: A Controlled Empirical Study
  2. 2026Iterative Multimodal Retrieval-Augmented Generation for Medical Question Answering
  3. 2025Beyond Retrieval: Ensembling Cross-Encoders and GPT Rerankers with LLMs for Biomedical QA
  4. 2026When Retrieval Doesn't Help: A Large-Scale Study of Biomedical RAG
  5. 2025Efficient and Reproducible Biomedical Question Answering using Retrieval Augmented Generation
  6. 2024BioRAGent: A Retrieval-Augmented Generation System for Showcasing Generative Query Expansion and Domain-Specific Search for Scientific Q&A
  7. 2025RAG-BioQA: A Retrieval-Augmented Generation Framework for Long-Form Biomedical Question Answering
  8. 2023RAMM: Retrieval-augmented Biomedical Visual Question Answering with Multi-modal Pre-training