mlsanityData to 7 Oct 2026

MIRA: A Novel Framework for Fusing Modalities in Medical RAG

arXiv 2507.07902 ↗First posted 10 Jul 2025cs.CVcited by 4

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have significantly advanced AI-assisted medical diagnosis, but they often generate factually inconsistent responses that deviate from established medical knowledge. Retrieval-Augmented Generation (RAG) enhances factual accuracy by integrating external sources, but it presents two key challenges. First, insufficient retrieval can miss critical information, whereas excessive retrieval can introduce irrelevant or misleading content, disrupting model output. Second, even when the model initially provides correct answers, over-reliance on retrieved data can lead to factual errors. To address these issues, we introduce the Multimodal Intelligent Retrieval and Augmentation (MIRA) framework, designed to optimize factual accuracy in MLLM. MIRA consists of two key components: (1) a calibrated Rethinking and Rearrangement module that dynamically adjusts the number of retrieved contexts to manage factual risk, and (2) A medical RAG framework integrating image embeddings and a medical knowledge base with a query-rewrite module for efficient multimodal reasoning. This enables the model to effectively integrate both its inherent knowledge and external references. Our evaluation of publicly available medical VQA and report generation benchmarks demonstrates that MIRA substantially enhances factual accuracy and overall performance, achieving new state-of-the-art results. Code is released at https://github.com/mbzuai-oryx/MIRA.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 17 of its 32 references that are in the map. First posted on arXiv 10 Jul 2025.

Builds on

17 of 32 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Models16 Oct
  2. 2024RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Models6 Jul
  3. 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale27 Jun
  4. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 13

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  5. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  6. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
  7. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  8. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 216
  9. 2023A Study of Generative Large Language Model for Medical Research and Healthcarecited by 160
  10. 2023Large language models in healthcare and medical domain: A reviewcited by 126
  11. 2021Medical Visual Question Answering: A Surveycited by 93
  12. 2023Q2ATransformer: Improving Medical VQA via an Answer Querying Decodercited by 31
1 more
  1. 2024FATE: Focal-modulated Attention Encoder for Multivariate Time-series Forecastingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025HeteroRAG: A Heterogeneous Retrieval-Augmented Generation Framework for Medical Vision Language Taskscited by 1
  2. 2026Fix Before Search: Benchmarking Agentic Query Visual Pre-processing in Multimodal Retrieval-augmented Generationcited by 1
  3. 2026Concept-Enhanced Multimodal RAG: Towards Interpretable and Accurate Radiology Report Generationcited by 0
  4. 2026QuPID: Quantum Parameter-Efficient Input-Dependent Retrieval Adaptation for Medical RAGcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Proactive Reasoning-with-Retrieval Framework for Medical Multimodal Large Language Models
  2. 2025Rethinking Retrieval-Augmented Generation for Medicine: A Large-Scale, Systematic Expert Evaluation and Practical Insights
  3. 2026Iterative Multimodal Retrieval-Augmented Generation for Medical Question Answering
  4. 2026Making medical vision-language models think causally across modalities with retrieval-augmented cross-modal reasoning
  5. 2025M-Eval: A Heterogeneity-Based Framework for Multi-evidence Validation in Medical RAG Systems
  6. 2025RAD: Towards Trustworthy Retrieval-Augmented Multi-modal Clinical Diagnosis
  7. 2026CURE: A Multimodal Benchmark for Clinical Understanding and Retrieval Evaluation
  8. 2025Multimodal Retrieval-Augmented Generation with Large Language Models for Medical VQA