mlsanityData to 14 Oct 2026

Augmenting Black-box LLMs with Medical Textbooks for Biomedical Question Answering

arXiv 2309.02233 ↗First posted 5 Sep 2023cs.CL cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Large-scale language models (LLMs) like ChatGPT have demonstrated impressive abilities in generating responses based on human instructions. However, their use in the medical field can be challenging due to their lack of specific, in-depth knowledge. In this study, we present a system called LLMs Augmented with Medical Textbooks (LLM-AMT) designed to enhance the proficiency of LLMs in specialized domains. LLM-AMT integrates authoritative medical textbooks into the LLMs' framework using plug-and-play modules. These modules include a Query Augmenter, a Hybrid Textbook Retriever, and a Knowledge Self-Refiner. Together, they incorporate authoritative medical knowledge. Additionally, an LLM Reader aids in contextual understanding. Our experimental results on three medical QA tasks demonstrate that LLMAMT significantly improves response quality, with accuracy gains ranging from 11.6% to 16.6%. Notably, with GPT-4-Turbo as the base model, LLM-AMT outperforms the specialized Med-PaLM 2 model pre-trained on a massive amount of medical corpus by 2-3%. We found that despite being 100x smaller in size, medical textbooks as a retrieval corpus is proven to be a more effective knowledge database than Wikipedia in the medical domain, boosting performance by 7.8%-13.7%.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 38 of its 54 references that are in the map. First posted on arXiv 5 Sep 2023.

Builds on

38 of 54 references.

References · 38

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  7. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  8. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  9. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  10. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  11. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  12. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
26 more
  1. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  2. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  3. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  4. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  5. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  6. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  7. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  8. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  9. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  10. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  11. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  12. 2023Augmented Language Models: a Surveycited by 403
  13. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 353
  14. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 286
  15. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 211
  16. 2022Can large language models reason about medical questions?cited by 194
  17. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  18. 2022A Survey on Retrieval-Augmented Text Generationcited by 158
  19. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  20. 2021A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniquescited by 69
  21. 2021Learning Passage Impacts for Inverted Indexescited by 68
  22. 2022Tevatron: An Efficient and Flexible Toolkit for Dense Retrievalcited by 47
  23. 2022Neural Approaches to Conversational Information Retrievalcited by 42
  24. 2018Medical Exam Question Answering with Large-scale Reading Comprehensioncited by 29
  25. 2021Sentence-aware Contrastive Learning for Open-Domain Passage Retrievalcited by 11
  26. 2022TegTok: Augmenting Text Generation via Task-specific and Open-world Knowledgecited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 3.

11 papers.

  1. 2024RARE: Retrieval-Augmented Reasoning Enhancement for Large Language Modelscited by 25
  2. 2025Towards Omni-RAG: Comprehensive Retrieval-Augmented Generation for Large Language Models in Medical Applicationscited by 13
  3. 2025Med-R2: Crafting Trustworthy LLM Physicians via Retrieval and Reasoning of Evidence-Based Medicinecited by 5
  4. 2025Queueing, Predictions, and LLMs: Challenges and Open Problemscited by 5
  5. 2025Medical Reasoning in LLMs: An In-Depth Analysis of DeepSeek R1cited by 5
  6. 2026Improving the Safety and Trustworthiness of Medical AI via Multi-Agent Evaluation Loopscited by 3
  7. 2025RAR2: Retrieval-Augmented Medical Reasoning via Thought-Driven Retrievalcited by 2
  8. 2026MedTutor: A Retrieval-Augmented LLM System for Case-Based Medical Educationcited by 2
  9. 2025POLYRAG: Integrating Polyviews into Retrieval-Augmented Generation for Medical Applicationscited by 1
  10. 2025Investigating LLM Capabilities on Long Context Comprehension for Medical Question Answeringcited by 0
  11. 2026Domain Fine-Tuning vs. Retrieval-Augmented Generation for Medical Multiple-Choice Question Answering: A Controlled Comparison at the 4B-Parameter Scalecited by 0