# RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Models

arXiv: https://arxiv.org/abs/2407.05131 · HTML: https://mlsanity.com/paper/2407.05131/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 6 Jul 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Medical vision-language models](https://mlsanity.com/mllm/medical-vision-language-models/)

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding](https://mlsanity.com/paper/2311.16922/) (2023)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation](https://mlsanity.com/paper/2311.17911/) (2023)
- [Med-Flamingo: a Multimodal Medical Few-shot Learner](https://mlsanity.com/paper/2307.15189/) (2023)
- [Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control](https://mlsanity.com/paper/2110.01052/) (2021)
- [PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documents](https://mlsanity.com/paper/2303.07240/) (2023)
- [Aligning Modalities in Vision Large Language Models via Preference Fine-tuning](https://mlsanity.com/paper/2402.11411/) (2024)
- [Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data](https://mlsanity.com/paper/2308.02463/) (2023)
- [FairCLIP: Harnessing Fairness in Vision-Language Learning](https://mlsanity.com/paper/2403.19949/) (2024)
- [GSCo: Towards Generalizable AI in Medicine via Generalist-Specialist Collaboration](https://mlsanity.com/paper/2404.15127/) (2024)
- [OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding](https://mlsanity.com/paper/2406.07471/) (2024)
- [RAMM: Retrieval-augmented Biomedical Visual Question Answering with Multi-modal Pre-training](https://mlsanity.com/paper/2303.00534/) (2023)
- [HGCLIP: Exploring Vision-Language Models with Graph Representations for Hierarchical Understanding](https://mlsanity.com/paper/2311.14064/) (2023)
- [NurViD: A Large Expert-Level Video Database for Nursing Procedure Activity Understanding](https://mlsanity.com/paper/2310.13347/) (2023)
- [Improving Medical Multi-modal Contrastive Learning with Expert Annotations](https://mlsanity.com/paper/2403.10153/) (2024)
- [LMPT: Prompt Tuning with Class-Specific Embedding Loss for Long-tailed Multi-Label Visual Recognition](https://mlsanity.com/paper/2305.04536/) (2023)
- [TP-DRSeg: Improving Diabetic Retinopathy Lesion Segmentation with Explicit Text-Prompts Assisted SAM](https://mlsanity.com/paper/2406.15764/) (2024)
- [Memory-based Cross-modal Semantic Alignment Network for Radiology Report Generation](https://mlsanity.com/paper/2404.00588/) (2024)
- [Generalizing to Unseen Domains in Diabetic Retinopathy with Disentangled Representations](https://mlsanity.com/paper/2406.06384/) (2024)

## Built on by (most cited first, 30 of 60)

- [AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases](https://mlsanity.com/paper/2407.12784/) (2024)
- [A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond](https://mlsanity.com/paper/2503.21614/) (2025)
- [MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Models](https://mlsanity.com/paper/2410.13085/) (2024)
- [VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning](https://mlsanity.com/paper/2505.22019/) (2025)
- [MDocAgent: A Multi-Modal Multi-Agent Framework for Document Understanding](https://mlsanity.com/paper/2503.13964/) (2025)
- [MMedAgent-RL: Optimizing Multi-Agent Collaboration for Multimodal Medical Reasoning](https://mlsanity.com/paper/2506.00555/) (2025)
- [M2IV: Towards Efficient and Fine-grained Multimodal In-Context Learning via Representation Engineering](https://mlsanity.com/paper/2504.04633/) (2025)
- [MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models](https://mlsanity.com/paper/2410.10139/) (2024)
- [Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlook](https://mlsanity.com/paper/2503.18016/) (2025)
- [Beyond Text: Optimizing RAG with Multimodal Inputs for Industrial Applications](https://mlsanity.com/paper/2410.21943/) (2024)
- [MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attacks](https://mlsanity.com/paper/2502.17832/) (2025)
- [Retrieval-Augmented Perception: High-Resolution Image Perception Meets Visual RAG](https://mlsanity.com/paper/2503.01222/) (2025)
- [MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization](https://mlsanity.com/paper/2412.06141/) (2024)
- [FinSage: A Multi-aspect RAG System for Financial Filings Question Answering](https://mlsanity.com/paper/2504.14493/) (2025)
- [Benchmarking Poisoning Attacks against Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.18543/) (2025)
- [A Survey on Computational Pathology Foundation Models: Datasets, Adaptation Strategies, and Evaluation Tasks](https://mlsanity.com/paper/2501.15724/) (2025)
- [Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generation](https://mlsanity.com/paper/2503.06254/) (2025)
- [MAGNET: A Multi-agent Framework for Finding Audio-Visual Needles by Reasoning over Multi-Video Haystacks](https://mlsanity.com/paper/2506.07016/) (2025)
- [OMGM: Orchestrate Multiple Granularities and Modalities for Efficient Multimodal Retrieval](https://mlsanity.com/paper/2505.07879/) (2025)
- [One Pic is All it Takes: Poisoning Visual Document Retrieval Augmented Generation with a Single Image](https://mlsanity.com/paper/2504.02132/) (2025)
- [A Multimodal Multi-Agent Framework for Radiology Report Generation](https://mlsanity.com/paper/2505.09787/) (2025)
- [Integrating Chain-of-Thought and Retrieval Augmented Generation Enhances Rare Disease Diagnosis from Clinical Notes](https://mlsanity.com/paper/2503.12286/) (2025)
- [MoMA: A Mixture-of-Multimodal-Agents Architecture for Enhancing Clinical Prediction Modelling](https://mlsanity.com/paper/2508.05492/) (2025)
- [Memory-Augmented Multimodal LLMs for Surgical VQA via Self-Contained Inquiry](https://mlsanity.com/paper/2411.10937/) (2024)
- [Mixture-of-Retrieval Experts for Reasoning-Guided Multimodal Knowledge Exploitation](https://mlsanity.com/paper/2505.22095/) (2025)
- [Med-VCD: Mitigating Hallucination for Medical Large Vision Language Models through Visual Contrastive Decoding](https://mlsanity.com/paper/2512.01922/) (2025)
- [MOTOR: Multimodal Optimal Transport via Grounded Retrieval in Medical Visual Question Answering](https://mlsanity.com/paper/2506.22900/) (2025)
- [Visual Alignment of Medical Vision-Language Models for Grounded Radiology Report Generation](https://mlsanity.com/paper/2512.16201/) (2025)
- [HoPE: Hybrid of Position Embedding for Long Context Vision-Language Models](https://mlsanity.com/paper/2505.20444/) (2025)
- [MIRA: A Novel Framework for Fusing Modalities in Medical RAG](https://mlsanity.com/paper/2507.07902/) (2025)

## Abstract (arXiv)

The recent emergence of Medical Large Vision Language Models (Med-LVLMs) has enhanced medical diagnosis. However, current Med-LVLMs frequently encounter factual issues, often generating responses that do not align with established medical facts. Retrieval-Augmented Generation (RAG), which utilizes external knowledge, can improve the factual accuracy of these models but introduces two major challenges. First, limited retrieved contexts might not cover all necessary information, while excessive retrieval can introduce irrelevant and inaccurate references, interfering with the model's generation. Second, in cases where the model originally responds correctly, applying RAG can lead to an over-reliance on retrieved contexts, resulting in incorrect answers. To address these issues, we propose RULE, which consists of two components. First, we introduce a provably effective strategy for controlling factuality risk through the calibrated selection of the number of retrieved contexts. Second, based on samples where over-reliance on retrieved contexts led to errors, we curate a preference dataset to fine-tune the model, balancing its dependence on inherent knowledge and retrieved contexts for generation. We demonstrate the effectiveness of RULE on medical VQA and report generation tasks across three datasets, achieving an average improvement of 47.4% in factual accuracy. We publicly release our benchmark and code in https://github.com/richard-peng-xia/RULE.