mlsanityData to 7 Oct 2026

A Survey of Large Language Models in Medicine: Progress, Application, and Challenge

arXiv 2311.05112 ↗First posted 9 Nov 2023cs.CL cs.AIcited by 123

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs), such as ChatGPT, have received substantial attention due to their capabilities for understanding and generating human language. While there has been a burgeoning trend in research focusing on the employment of LLMs in supporting different medical tasks (e.g., enhancing clinical diagnostics and providing medical education), a review of these efforts, particularly their development, practical applications, and outcomes in medicine, remains scarce. Therefore, this review aims to provide a detailed overview of the development and deployment of LLMs in medicine, including the challenges and opportunities they face. In terms of development, we provide a detailed introduction to the principles of existing medical LLMs, including their basic model structures, number of parameters, and sources and scales of data used for model development. It serves as a guide for practitioners in developing medical LLMs tailored to their specific needs. In terms of deployment, we offer a comparison of the performance of different LLMs across various medical tasks, and further compare them with state-of-the-art lightweight models, aiming to provide an understanding of the advantages and limitations of LLMs in medicine. Overall, in this review, we address the following questions: 1) What are the practices for developing medical LLMs 2) How to measure the medical task performance of LLMs in a medical setting? 3) How have medical LLMs been employed in real-world practice? 4) What challenges arise from the use of medical LLMs? and 5) How to more effectively develop and deploy medical LLMs? By answering these questions, this review aims to provide insights into the opportunities for LLMs in medicine and serve as a practical resource. We also maintain a regularly updated list of practical guides on medical LLMs at https://github.com/AI-in-Health/MedLLMsPracticalGuide

Key facts. Cited by 123 papers in the map (data through 7 Oct 2026). Builds on 220 of its 432 references that are in the map. First posted on arXiv 9 Nov 2023.

Builds on

220 of 432 references.

References · 220

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  9. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  10. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  11. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  12. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
38 more
  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  12. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  13. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  14. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  15. 2023Qwen Technical Reportcited by 3,273
  16. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  17. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  18. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  19. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  20. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  21. 2023A Survey of Large Language Modelscited by 2,537
  22. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  23. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  24. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  25. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  26. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  27. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  28. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  29. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  30. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  31. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  32. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  33. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  34. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  35. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  36. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  37. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  38. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324

50 most cited of 220 All 220 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 54; 2025: 47; 2026: 21.

123 papers.

Sort:

By area:

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  2. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  3. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 125
  4. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 121
  5. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 113
  6. 2025HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptationcited by 112
  7. 2024AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulatorcited by 99
  8. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  9. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  10. 2024Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questionscited by 54
  11. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicinecited by 45
  12. 2024How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?cited by 44
38 more
  1. 2024Autoregressive Models in Vision: A Surveycited by 39
  2. 2024Large Language Models for Medicine: A Surveycited by 38
  3. 2024Automatic Interactive Evaluation for Large Language Models with State Aware Patient Simulatorcited by 36
  4. 2025UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Makingcited by 36
  5. 2024Large Language Models in Mental Health Care: a Scoping Reviewcited by 34
  6. 2024LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Modelscited by 31
  7. 2024CS-Bench: A Comprehensive Benchmark for Large Language Models towards Computer Science Masterycited by 30
  8. 2024ClinicalLab: Aligning Agents for Multi-Departmental Clinical Diagnostics in the Real Worldcited by 28
  9. 2025Fine-tuning Large Language Models for Improving Factuality in Legal Question Answeringcited by 28
  10. 2025ALFA: Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoningcited by 24
  11. 2025Primus: A Pioneering Collection of Open-Source Datasets for Cybersecurity LLM Trainingcited by 23
  12. 2024A Survey of Conversational Searchcited by 22
  13. 2025Seed-X: Building Strong Multilingual Translation LLM with 7B Parameterscited by 21
  14. 2024Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQAcited by 19
  15. 2025Automatic Evaluation of Healthcare LLMs Beyond Question-Answeringcited by 19
  16. 2023Distilling Large Language Models for Matching Patients to Clinical Trialscited by 16
  17. 2024CLST: Cold-Start Mitigation in Knowledge Tracing by Aligning a Generative Language Model as a Students' Knowledge Tracercited by 16
  18. 2025Citrus: Leveraging Expert Cognitive Pathways in a Medical Language Model for Advanced Medical Decision Supportcited by 16
  19. 2024To Generate or to Retrieve? On the Effectiveness of Artificial Contexts for Medical Open-Domain Question Answeringcited by 15
  20. 2024ProFLingo: A Fingerprinting-based Intellectual Property Protection Scheme for Large Language Modelscited by 15
  21. 2025Towards Omni-RAG: Comprehensive Retrieval-Augmented Generation for Large Language Models in Medical Applicationscited by 13
  22. 2026PLawBench: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practicecited by 13
  23. 2024Large Language Models in the Clinic: A Comprehensive Benchmarkcited by 12
  24. 2024Evaluating LLM Reasoning in the Operations Research Domain with ORQAcited by 12
  25. 2025Agentic Medical Knowledge Graphs Enhance Medical Question Answering: Bridging the Gap Between LLMs and Evolving Medical Knowledgecited by 12
  26. 2024A Scoping Review of Deep Learning Methods for Photoplethysmography Datacited by 11
  27. 2024Advancing Healthcare Automation: Multi-Agent System for Medical Necessity Justificationcited by 10
  28. 2024FoodSky: A Food-oriented Large Language Model that Passes the Chef and Dietetic Examinationcited by 10
  29. 2025Adaptive Distraction: Probing LLM Contextual Robustness with Automated Tree Searchcited by 10
  30. 2025Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applicationscited by 10
  31. 2024Towards Adapting Open-Source Large Language Models for Expert-Level Clinical Note Generationcited by 9
  32. 2025Uncertainty-Aware Large Language Models for Explainable Disease Diagnosiscited by 9
  33. 2024Geneverse: A collection of Open-source Multimodal Large Language Models for Genomic and Proteomic Researchcited by 8
  34. 2025OntoTune: Ontology-Driven Self-training for Aligning Large Language Modelscited by 8
  35. 2025A Survey of Inductive Reasoning for Large Language Modelscited by 8
  36. 2024Aligning Large Language Models for Enhancing Psychiatric Interviews Through Symptom Delineation and Summarization: Pilot Studycited by 7
  37. 2024RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in Large Vision Language Modelscited by 7
  38. 2024Improving Clinical Note Generation from Complex Doctor-Patient Conversationcited by 7

50 most cited of 123 All 123 (JSON)