mlsanityData to 7 Oct 2026

JMLR: Joint Medical LLM and Retrieval Training for Enhancing Reasoning and Professional Question Answering Capability

arXiv 2402.17887 ↗First posted 27 Feb 2024cs.CL cs.IRcited by 32

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated a remarkable potential in medical knowledge acquisition and question-answering. However, LLMs can potentially hallucinate and yield factually incorrect outcomes, even with domain-specific pretraining. Previously, retrieval augmented generation (RAG) has limited success in addressing hallucinations. Unlike previous methods in RAG where the retrieval model was trained separately from the LLM, we introduce JMLR (for Jointly trains LLM and information Retrieval) during the fine-tuning phase. The synchronized training mechanism enhances JMLR's ability to retrieve clinical guidelines and leverage medical knowledge to reason and answer questions and reduces the demand for computational resources. We evaluated JMLR on the important medical question-answering application. Our experimental results demonstrate that JMLR-13B (70.5%) outperforms a previous state-of-the-art open-source model using conventional pre-training and fine-tuning Meditron-70B (68.9%) and Llama2-13B with RAG (67.7%) on a medical question-answering dataset. Comprehensive evaluations reveal JMLR-13B enhances reasoning quality and reduces hallucinations better than Claude3-Opus. Additionally, JMLR-13B (148 GPU hours) also trains much faster than Meditron-70B (42630 GPU hours). Through this work, we provide a new and efficient knowledge enhancement method for healthcare, demonstrating the potential of integrating retrieval and LLM training for medical question-answering systems.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 47 of its 73 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

47 of 73 references.

References · 47

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  9. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  10. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  11. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  12. 2023Code Llama: Open Foundation Models for Codecited by 2,105
35 more
  1. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  2. 2019Language Models as Knowledge Bases?cited by 1,922
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  4. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  5. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  6. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  7. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  8. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  9. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  10. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  11. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  12. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  13. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  14. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  15. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  16. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  17. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  18. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 299
  19. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 282
  20. 2023HuatuoGPT, towards Taming Language Model to Be a Doctorcited by 211
  21. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 209
  22. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  23. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  24. 2023A Study of Generative Large Language Model for Medical Research and Healthcarecited by 160
  25. 2022Deep Bidirectional Language-Knowledge Graph Pretrainingcited by 156
  26. 2023Almanac: Retrieval-Augmented Language Models for Clinical Medicinecited by 122
  27. 2023Lift Yourself Up: Retrieval-augmented Text Generation with Self Memorycited by 103
  28. 2024Aligner: Efficient Alignment by Learning to Correctcited by 96
  29. 2023PMC-LLaMA: Towards Building Open-source Language Models for Medicinecited by 87
  30. 2021Can Language Models be Biomedical Knowledge Bases?cited by 83
  31. 2023Publicly Shareable Clinical Large Language Model Built on Synthetic Clinical Notescited by 51
  32. 2023NoteChat: A Dataset of Synthetic Doctor-Patient Conversations Conditioned on Clinical Notescited by 48
  33. 2023Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domaincited by 38
  34. 2023Creating Trustworthy LLMs: Dealing with Hallucinations in Healthcare AIcited by 33
  35. 2022Context Variance Evaluation of Pretrained Language Models for Prompt-based Biomedical Knowledge Probingcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 13; 2026: 3.

32 papers.

Sort:

By area:

  1. 2024BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrieverscited by 38
  2. 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
  3. 2024TWIN-GPT: Digital Twins for Clinical Trials via Large Language Modelcited by 24
  4. 2024DrHouse: An LLM-empowered Diagnostic Reasoning System through Harnessing Outcomes from Sensor Data and Expert Knowledgecited by 24
  5. 2024RARE: Retrieval-Augmented Reasoning Enhancement for Large Language Modelscited by 24
  6. 2024A Survey of Conversational Searchcited by 22
  7. 2024A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluationscited by 16
  8. 2024MCQG-SRefine: Multiple Choice Question Generation and Evaluation with Iterative Self-Critique, Correction, and Comparison Feedbackcited by 14
  9. 2026Knowing When to Abstain: Medical LLMs Under Clinical Uncertaintycited by 13
  10. 2025From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculationscited by 11
  11. 2025ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidancecited by 10
  12. 2024LLMs are not Zero-Shot Reasoners for Biomedical Information Extractioncited by 9
20 more
  1. 2024From Text to Multimodality: Exploring the Evolution and Impact of Large Language Models in Medical Practicecited by 8
  2. 2025The Multi-Round Diagnostic RAG Framework for Emulating Clinical Reasoningcited by 8
  3. 2025MedEthicsQA: A Comprehensive Question Answering Benchmark for Medical Ethics Evaluation of LLMscited by 8
  4. 2025Clinical-R1: Empowering Large Language Models for Faithful and Comprehensive Reasoning with Clinical Objective Relative Policy Optimizationcited by 6
  5. 2024Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answeringcited by 5
  6. 2025Medical Reasoning in LLMs: An In-Depth Analysis of DeepSeek R1cited by 5
  7. 2025EHRStruct: A Comprehensive Benchmark Framework for Evaluating Large Language Models on Structured Electronic Health Record Taskscited by 5
  8. 2024Generating a Low-code Complete Workflow via Task Decomposition and RAGcited by 3
  9. 2025Generative AI for Healthcare: Fundamentals, Challenges, and Perspectivescited by 3
  10. 2024Watermarking Large Language Models and the Generated Content: Opportunities and Challengescited by 2
  11. 2025Conversation Forests: The Key to Fine Tuning Large Language Models for Multi-Turn Medical Conversations is Branchingcited by 2
  12. 2025Med-R3: Enhancing Medical Retrieval-Augmented Reasoning of LLMs via Progressive Reinforcement Learningcited by 2
  13. 2024SleepCoT: A Lightweight Personalized Sleep Health Model via Chain-of-Thought Distillationcited by 1
  14. 2024Language Models And A Second Opinion Use Case: The Pocket Professionalcited by 1
  15. 2024GuidelineGuard: An Agentic Framework for Medical Note Evaluation with Guideline Adherencecited by 0
  16. 2025Lifelong Reinforcement Learning with Similarity-Driven Weighting by Large Modelscited by 0
  17. 2025Beyond Retrieval: Joint Supervision and Multimodal Document Ranking for Textbook Question Answeringcited by 0
  18. 2025HIVMedQA: Benchmarking large language models for HIV medical decision supportcited by 0
  19. 2026Do No Harm? Hallucination and Actor-Level Abuse in Web-Deployed Medical Large Language Modelscited by 0
  20. 2026Large language models in medical time series analysiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answering
  2. 2025Optimizing Medical Question-Answering Systems: A Comparative Study of Fine-Tuned and Zero-Shot Large Language Models with RAG Framework
  3. 2026A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering
  4. 2024Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questions
  5. 2026Domain Fine-Tuning vs. Retrieval-Augmented Generation for Medical Multiple-Choice Question Answering: A Controlled Comparison at the 4B-Parameter Scale
  6. 2025Investigating LLM Capabilities on Long Context Comprehension for Medical Question Answering
  7. 2025RAR2: Retrieval-Augmented Medical Reasoning via Thought-Driven Retrieval
  8. 2024Tool Calling: Enhancing Medication Consultation via Retrieval-Augmented Large Language Models