mlsanityData to 7 Oct 2026

Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questions

arXiv 2408.00727 ↗First posted 1 Aug 2024cs.CL cs.AIcited by 54

On the map: Retrieval-augmented generation → Medical and biomedical RAG

Abstract (arXiv)

The emergent abilities of large language models (LLMs) have demonstrated great potential in solving medical questions. They can possess considerable medical knowledge, but may still hallucinate and are inflexible in the knowledge updates. While Retrieval-Augmented Generation (RAG) has been proposed to enhance the medical question-answering capabilities of LLMs with external knowledge bases, it may still fail in complex cases where multiple rounds of information-seeking are required. To address such an issue, we propose iterative RAG for medicine (i-MedRAG), where LLMs can iteratively ask follow-up queries based on previous information-seeking attempts. In each iteration of i-MedRAG, the follow-up queries will be answered by a conventional RAG system and they will be further used to guide the query generation in the next iteration. Our experiments show the improved performance of various LLMs brought by i-MedRAG compared with conventional RAG on complex questions from clinical vignettes in the United States Medical Licensing Examination (USMLE), as well as various knowledge tests in the Massive Multitask Language Understanding (MMLU) dataset. Notably, our zero-shot i-MedRAG outperforms all existing prompt engineering and fine-tuning methods on GPT-3.5, achieving an accuracy of 69.68% on the MedQA dataset. In addition, we characterize the scaling properties of i-MedRAG with different iterations of follow-up queries and different numbers of queries per iteration. Our case studies show that i-MedRAG can flexibly ask follow-up queries to form reasoning chains, providing an in-depth analysis of medical questions. To the best of our knowledge, this is the first-of-its-kind study on incorporating follow-up queries into medical RAG. The implementation of i-MedRAG is available at https://github.com/Teddy-XiongGZ/MedRAG.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation. Builds on 40 of its 45 references that are in the map. First posted on arXiv 1 Aug 2024.

Builds on

40 of 45 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Benchmarking Retrieval-Augmented Generation for Medicine20 Feb
  2. 2023Clinfo.ai: An Open-Source Retrieval-Augmented Large Language Model System for Answering Medical Questions using Scientific Literature24 Oct
  3. 2023Almanac: Retrieval-Augmented Language Models for Clinical Medicine1 Mar

Other references · 37

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  7. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  8. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  9. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  10. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  11. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  12. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
25 more
  1. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  2. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
  3. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 319
  4. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  5. 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 244
  6. 2024Capabilities of Gemini Models in Medicinecited by 231
  7. 2022Can large language models reason about medical questions?cited by 194
  8. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 182
  9. 2023Self-Alignment with Instruction Backtranslationcited by 149
  10. 2024Continual Learning for Large Language Models: A Surveycited by 144
  11. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
  12. 2023Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Healthcited by 118
  13. 2024BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Textcited by 85
  14. 2023Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicinecited by 83
  15. 2023Matching Patients to Clinical Trials with Large Language Modelscited by 68
  16. 2021Biomedical Question Answering: A Survey of Approaches and Challengescited by 46
  17. 2023Summarizing, Simplifying, and Synthesizing Medical Evidence Using GPT-3 (with Varying Success)cited by 40
  18. 2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approachcited by 40
  19. 2023Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphscited by 35
  20. 2024Zero-Shot Clinical Trial Patient Matching with LLMscited by 35
  21. 2024MedAdapter: Efficient Test-Time Adaptation of Large Language Models towards Medical Reasoningcited by 35
  22. 2023Scaling Clinical Trial Matching Using Large Language Models: A Case Study in Oncologycited by 24
  23. 2023PubMed and Beyond: Biomedical Literature Search in the Age of Artificial Intelligencecited by 16
  24. 2024Team IELAB at TREC Clinical Trial Track 2023: Enhancing Clinical Trial Retrieval with Neural Rankers and Large Language Modelscited by 3
  25. 2024Answering real-world clinical questions using large language model based systemscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 25; 2026: 23.

54 papers.

Sort:

By area:

  1. 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agentscited by 49
  2. 2025m1: Unleash the Potential of Test-Time Scaling for Medical Reasoning with Large Language Modelscited by 42
  3. 2024Rationale-Guided Retrieval Augmented Generation for Medical Question Answeringcited by 41
  4. 2024RARE: Retrieval-Augmented Reasoning Enhancement for Large Language Modelscited by 24
  5. 2025Towards Omni-RAG: Comprehensive Retrieval-Augmented Generation for Large Language Models in Medical Applicationscited by 13
  6. 2025MedCite: Can Language Models Generate Verifiable Text for Medicine?cited by 11
  7. 2025From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculationscited by 11
  8. 2025GPIoT: Tailoring Small Language Models for IoT Program Synthesis and Developmentcited by 10
  9. 2025Out of Style: RAG's Fragility to Linguistic Variationcited by 10
  10. 2025SearchRAG: Can Search Engines Be Helpful for LLM-based Medical Question Answering?cited by 9
  11. 2025Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Modelscited by 7
  12. 2025Multi-step retrieval and reasoning improves radiology question answering with large language modelscited by 7
38 more
  1. 2025Med-R2: Crafting Trustworthy LLM Physicians via Retrieval and Reasoning of Evidence-Based Medicinecited by 5
  2. 2025RGAR: Recurrence Generation-augmented Retrieval for Factual-aware Medical Question Answeringcited by 5
  3. 2025Expert-Guided Prompting and Retrieval-Augmented Generation for Emergency Medical Service Question Answeringcited by 5
  4. 2024HEALTH-PARIKSHA: Assessing RAG Models for Health Chatbots in Real-World Multilingual Settingscited by 4
  5. 2025DoctorRAG: Medical RAG Fusing Knowledge with Patient Analogy through Textual Gradientscited by 4
  6. 2026TARSE: Test-Time Adaptation via Retrieval of Skills and Experience for Reasoning Agentscited by 4
  7. 2024Entry-level guide to the use of large language models for medical researchcited by 3
  8. 2024Multi-hop Evidence Pursuit Meets the Web: Team Papelo at FEVER 2024cited by 3
  9. 2025Progressive Multimodal Search and Reasoning for Knowledge-Intensive Visual Question Answeringcited by 3
  10. 2025RAR2: Retrieval-Augmented Medical Reasoning via Thought-Driven Retrievalcited by 2
  11. 2026CT Open: An Open-Access, Uncontaminated, Live Platform for the Open Challenge of Clinical Trial Outcome Predictioncited by 2
  12. 2025POLYRAG: Integrating Polyviews into Retrieval-Augmented Generation for Medical Applicationscited by 1
  13. 2025CLARIFY: A Specialist-Generalist Framework for Accurate and Lightweight Dermatological Visual Question Answeringcited by 1
  14. 2025Build AI Assistants using Large Language Models and Agents to Enhance the Engineering Education of Biomechanicscited by 1
  15. 2025A Medical Multimodal Diagnostic Framework Integrating Vision-Language Models and Logic Tree Reasoningcited by 1
  16. 2026When Iterative RAG Beats Ideal Evidence: A Diagnostic Study in Scientific Multi-hop Question Answeringcited by 1
  17. 2026MED-COPILOT: A Medical Assistant Powered by GraphRAG and Similar Patient Case Retrievalcited by 1
  18. 2026IterCOMP: Reasoning-aware Adaptive Prompt Compression for Multi-hop Question Answeringcited by 1
  19. 2024GuidelineGuard: An Agentic Framework for Medical Note Evaluation with Guideline Adherencecited by 0
  20. 2025Optimizing Retrieval-Augmented Generation for Electrical Engineering: A Case Study on ABB Circuit Breakerscited by 0
  21. 2025MedVQA-TREE: A Multimodal Reasoning and Retrieval Framework for Sarcopenia Predictioncited by 0
  22. 2025From Retrieval to Generation: Unifying External and Parametric Knowledge for Medical Question Answeringcited by 0
  23. 2025Biomedical Hypothesis Explainability with Graph-Based Context Retrievalcited by 0
  24. 2025Large Language Model Aided Birt-Hogg-Dube Syndrome Diagnosis with Multimodal Retrieval-Augmented Generationcited by 0
  25. 2026Self-MedRAG: a Self-Reflective Hybrid Retrieval-Augmented Generation Framework for Reliable Medical Question Answeringcited by 0
  26. 2026TCM-DiffRAG: Personalized Syndrome Differentiation Reasoning Method for Traditional Chinese Medicine based on Knowledge Graph and Chain of Thoughtcited by 0
  27. 2026Hypothesis-Conditioned Query Rewriting for Decision-Useful Retrievalcited by 0
  28. 2026From Exposure to Internalization: Dual-Stream Calibration for In-context Clinical Reasoningcited by 0
  29. 2026Effects of Cross-lingual Evidence in Multilingual Medical Question Answeringcited by 0
  30. 2026HealthNLP_Retrievers at ArchEHR-QA 2026: Cascaded LLM Pipeline for Grounded Clinical Question Answeringcited by 0
  31. 2026Iterative Multimodal Retrieval-Augmented Generation for Medical Question Answeringcited by 0
  32. 2026ReLay: Personalized LLM-Generated Plain-Language Summaries for Better Understanding, but at What Cost?cited by 0
  33. 2026Mind Companion: An Embodied Conversational Agent for Process-Based Psychotherapycited by 0
  34. 2026Configurable Clinical Information Extraction with Agentic RAG: What Works, What Breaks, and Whycited by 0
  35. 2026Hybrid-IR: Dual-Path Hybrid Retrieval with Iterative Reasoning for Complex Medical Question Answeringcited by 0
  36. 2026Retrieval-Augmented Generation to Support Railways Engineering Tasks: A Case Studycited by 0
  37. 2026SSE-Bio: A Structured Self-Evolving Agent with Agentic Retrieval Policy for Multi-Hop Biomedical Reasoningcited by 0
  38. 2026MedAgent-R1: Faithfulness-Aware Reinforcement Learning for Evidence-Grounded Medical Reasoningcited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answering
  2. 2026A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering
  3. 2024Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answering
  4. 2025Optimizing Medical Question-Answering Systems: A Comparative Study of Fine-Tuned and Zero-Shot Large Language Models with RAG Framework
  5. 2025Retrieval-Augmented Generation in Medicine: A Scoping Review of Technical Implementations, Clinical Applications, and Ethical Considerations
  6. 2024Tool Calling: Enhancing Medication Consultation via Retrieval-Augmented Large Language Models
  7. 2024Development and Testing of Retrieval Augmented Generation in Large Language Models -- A Case Study Report
  8. 2024The Geometry of Queries: Query-Based Innovations in Retrieval-Augmented Generation for Healthcare QA