# Rationale-Guided Retrieval Augmented Generation for Medical Question Answering

arXiv: https://arxiv.org/abs/2411.00300 · HTML: https://mlsanity.com/paper/2411.00300/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Placed in

- Retrieval-augmented generation / [Medical and biomedical RAG](https://mlsanity.com/rag/medical-and-biomedical-rag/)

## Builds on (most cited first, 28 of 28)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity](https://mlsanity.com/paper/2403.14403/) (2024)
- [Benchmarking Retrieval-Augmented Generation for Medicine](https://mlsanity.com/paper/2402.13178/) (2024)
- [BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains](https://mlsanity.com/paper/2402.10373/) (2024)
- [ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2311.09476/) (2023)
- [RAFT: Adapting Language Model to Domain Specific RAG](https://mlsanity.com/paper/2403.10131/) (2024)
- [RealTime QA: What's the Answer Right Now?](https://mlsanity.com/paper/2207.13332/) (2022)
- [Capabilities of Gemini Models in Medicine](https://mlsanity.com/paper/2404.18416/) (2024)
- [MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrieval](https://mlsanity.com/paper/2307.00589/) (2023)
- [Almanac: Retrieval-Augmented Language Models for Clinical Medicine](https://mlsanity.com/paper/2303.01229/) (2023)
- [ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence](https://mlsanity.com/paper/2404.10198/) (2024)
- [Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting](https://mlsanity.com/paper/2407.08223/) (2024)
- [RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generation](https://mlsanity.com/paper/2403.05313/) (2024)
- [Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks](https://mlsanity.com/paper/2305.18395/) (2023)
- [Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questions](https://mlsanity.com/paper/2408.00727/) (2024)
- [Neural Retrievers are Biased Towards LLM-Generated Content](https://mlsanity.com/paper/2310.20501/) (2023)
- [Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLP](https://mlsanity.com/paper/2106.06830/) (2021)
- [Small Language Models Learn Enhanced Reasoning Skills from Medical Textbooks](https://mlsanity.com/paper/2404.00376/) (2024)
- [Similarity is Not All You Need: Endowing Retrieval Augmented Generation with Multi Layered Thoughts](https://mlsanity.com/paper/2405.19893/) (2024)
- [Bailicai: A Domain-Optimized Retrieval-Augmented Generation Framework for Medical Applications](https://mlsanity.com/paper/2407.21055/) (2024)
- [Automatic Creation of Named Entity Recognition Datasets by Querying Phrase Representations](https://mlsanity.com/paper/2210.07586/) (2022)

## Built on by (most cited first, 30 of 41)

- [Training a Utility-based Retriever Through Shared Context Attribution for Retrieval-Augmented Language Models](https://mlsanity.com/paper/2504.00573/) (2025)
- [Understanding the Impact of Confidence in Retrieval Augmented Generation: A Case Study in the Medical Domain](https://mlsanity.com/paper/2412.20309/) (2024)
- [SearchRAG: Can Search Engines Be Helpful for LLM-based Medical Question Answering?](https://mlsanity.com/paper/2502.13233/) (2025)
- [Principled Context Engineering for RAG: Statistical Guarantees via Conformal Prediction](https://mlsanity.com/paper/2511.17908/) (2025)
- [Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domains](https://mlsanity.com/paper/2505.16014/) (2025)
- [MedCoT-RAG: Causal Chain-of-Thought RAG for Medical Question Answering](https://mlsanity.com/paper/2508.15849/) (2025)
- [Knowledge-Augmented Multimodal Clinical Rationale Generation for Disease Diagnosis with Small Language Models](https://mlsanity.com/paper/2411.07611/) (2024)
- [CLI-RAG: A Retrieval-Augmented Framework for Clinically Structured and Context Aware Text Generation with LLMs](https://mlsanity.com/paper/2507.06715/) (2025)
- [Architecting Clinical Collaboration: Multi-Agent Reasoning Systems for Multimodal Medical VQA](https://mlsanity.com/paper/2507.05520/) (2025)
- [RAR2: Retrieval-Augmented Medical Reasoning via Thought-Driven Retrieval](https://mlsanity.com/paper/2509.22713/) (2025)
- [POLYRAG: Integrating Polyviews into Retrieval-Augmented Generation for Medical Applications](https://mlsanity.com/paper/2504.14917/) (2025)
- [Continually Self-Improving Language Models for Bariatric Surgery Question--Answering](https://mlsanity.com/paper/2505.16102/) (2025)
- [Evaluating LLMs in Medicine: A Call for Rigor, Transparency](https://mlsanity.com/paper/2507.08916/) (2025)
- [MED-COPILOT: A Medical Assistant Powered by GraphRAG and Similar Patient Case Retrieval](https://mlsanity.com/paper/2603.00460/) (2026)
- [When Chain-of-Thought Backfires: Evaluating Prompt Sensitivity in Medical Language Models](https://mlsanity.com/paper/2603.25960/) (2026)
- [Process Reward Agents for Steering Knowledge-Intensive Reasoning](https://mlsanity.com/paper/2604.09482/) (2026)
- [REPAIR: Resolving Long-Tail Confusion in Scientific Retrievers via Fact-Verified Iterative Refinement](https://mlsanity.com/paper/2609.18262/) (2026)
- [ICA-RAG: Information Completeness Guided Adaptive Retrieval-Augmented Generation for Disease Diagnosis](https://mlsanity.com/paper/2502.14614/) (2025)
- [Accelerating Causal Network Discovery of Alzheimer Disease Biomarkers via Scientific Literature-based Retrieval Augmented Generation](https://mlsanity.com/paper/2504.08768/) (2025)
- [GraphMed-LT: Patient-Specific Graph Memory with Latent Clinical Thought Refinement for Multi-Turn Medical Conversations](https://mlsanity.com/paper/2510.03536/) (2025)
- [From Retrieval to Generation: Unifying External and Parametric Knowledge for Medical Question Answering](https://mlsanity.com/paper/2510.18297/) (2025)
- [Investigating LLM Capabilities on Long Context Comprehension for Medical Question Answering](https://mlsanity.com/paper/2510.18691/) (2025)
- [From Evidence-Based Medicine to Knowledge Graph: Retrieval-Augmented Generation for Sports Rehabilitation and a Domain Benchmark](https://mlsanity.com/paper/2601.00216/) (2026)
- [ExDR: Explanation-driven Dynamic Retrieval Enhancement for Multimodal Fake News Detection](https://mlsanity.com/paper/2601.15820/) (2026)
- [MedCoRAG: Interpretable Hepatology Diagnosis via Hybrid Evidence Retrieval and Multispecialty Consensus](https://mlsanity.com/paper/2603.05129/) (2026)
- [Ruling Out to Rule In: Contrastive Hypothesis Retrieval for Medical Question Answering](https://mlsanity.com/paper/2604.04593/) (2026)
- [Effects of Cross-lingual Evidence in Multilingual Medical Question Answering](https://mlsanity.com/paper/2604.20531/) (2026)
- [Iterative Multimodal Retrieval-Augmented Generation for Medical Question Answering](https://mlsanity.com/paper/2604.27724/) (2026)
- [Overview of the MedHopQA track at BioCreative IX: track description, participation and evaluation of systems for multi-hop medical question answering](https://mlsanity.com/paper/2605.12313/) (2026)
- [EHR-RAGp: Retrieval-Augmented Prototype-Guided Foundation Model for Electronic Health Records](https://mlsanity.com/paper/2605.12335/) (2026)

## Abstract (arXiv)

Large language models (LLM) hold significant potential for applications in biomedicine, but they struggle with hallucinations and outdated knowledge. While retrieval-augmented generation (RAG) is generally employed to address these issues, it also has its own set of challenges: (1) LLMs are vulnerable to irrelevant or incorrect context, (2) medical queries are often not well-targeted for helpful information, and (3) retrievers are prone to bias toward the specific source corpus they were trained on. In this study, we present RAG2 (RAtionale-Guided RAG), a new framework for enhancing the reliability of RAG in biomedical contexts. RAG2 incorporates three key innovations: a small filtering model trained on perplexity-based labels of rationales, which selectively augments informative snippets of documents while filtering out distractors; LLM-generated rationales as queries to improve the utility of retrieved snippets; a structure designed to retrieve snippets evenly from a comprehensive set of four biomedical corpora, effectively mitigating retriever bias. Our experiments demonstrate that RAG2 improves the state-of-the-art LLMs of varying sizes, with improvements of up to 6.1%, and it outperforms the previous best medical RAG model by up to 5.6% across three medical question-answering benchmarks. Our code is available at https://github.com/dmis-lab/RAG2.