mlsanityData to 7 Oct 2026

A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics

arXiv 2310.05694 ↗First posted 9 Oct 2023cs.CLcited by 162

Not on the map. Why?

Abstract (arXiv)

The utilization of large language models (LLMs) in the Healthcare domain has generated both excitement and concern due to their ability to effectively respond to freetext queries with certain professional knowledge. This survey outlines the capabilities of the currently developed LLMs for Healthcare and explicates their development process, with the aim of providing an overview of the development roadmap from traditional Pretrained Language Models (PLMs) to LLMs. Specifically, we first explore the potential of LLMs to enhance the efficiency and effectiveness of various Healthcare applications highlighting both the strengths and limitations. Secondly, we conduct a comparison between the previous PLMs and the latest LLMs, as well as comparing various LLMs with each other. Then we summarize related Healthcare training data, training methods, optimization strategies, and usage. Finally, the unique concerns associated with deploying LLMs in Healthcare settings are investigated, particularly regarding fairness, accountability, transparency and ethics. Our survey provide a comprehensive investigation from perspectives of both computer science and Healthcare specialty. Besides the discussion about Healthcare concerns, we supports the computer science community by compiling a collection of open source resources, such as accessible datasets, the latest methodologies, code implementations, and evaluation benchmarks in the Github. Summarily, we contend that a significant paradigm shift is underway, transitioning from PLMs to LLMs. This shift encompasses a move from discriminative AI approaches to generative AI approaches, as well as a shift from model-centered methodologies to data-centered methodologies. Also, we determine that the biggest obstacle of using LLMs in Healthcare are fairness, accountability, transparency and ethics.

Key facts. Cited by 162 papers in the map (data through 7 Oct 2026). Builds on 222 of its 421 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

222 of 421 references.

References · 222

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2017Proximal Policy Optimization Algorithmscited by 14,441
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  9. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  10. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  11. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  12. 2021Evaluating Large Language Models Trained on Codecited by 8,053
38 more
  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  13. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  14. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  15. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  16. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  17. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  18. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  19. 2023A Survey of Large Language Modelscited by 2,537
  20. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  21. 2018Deep contextualized word representationscited by 2,285
  22. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  23. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  24. 2019Language Models as Knowledge Bases?cited by 1,922
  25. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  26. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  27. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  28. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  29. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  30. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  31. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  32. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  33. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  34. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  35. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  36. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  37. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  38. 2023PaLM 2 Technical Reportcited by 1,154

50 most cited of 222 All 222 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 71; 2025: 57; 2026: 27.

162 papers.

Sort:

By area:

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  2. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  3. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 299
  4. 2023Efficient Large Language Models: A Surveycited by 183
  5. 2023Large language models in healthcare and medical domain: A reviewcited by 126
  6. 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 125
  7. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
  8. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 121
  9. 2024Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directionscited by 77
  10. 2024Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Datacited by 65
  11. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicinecited by 45
  12. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
38 more
  1. 2024Are LLMs Ready for Real-World Materials Discovery?cited by 40
  2. 2024FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answeringcited by 38
  3. 2025GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Imagescited by 35
  4. 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
  5. 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoningcited by 31
  6. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 29
  7. 2024WordGame: Efficient & Effective LLM Jailbreak via Simultaneous Obfuscation in Query and Responsecited by 28
  8. 2024Navigating LLM Ethics: Advancements, Challenges, and Future Directionscited by 28
  9. 2024Data-Centric Foundation Models in Computational Healthcare: A Surveycited by 27
  10. 2024Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentationcited by 24
  11. 2024Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Modelscited by 23
  12. 2024T-RAG: Lessons from the LLM Trenchescited by 23
  13. 2025Attention Reallocation: Towards Zero-cost and Controllable Hallucination Mitigation of MLLMscited by 23
  14. 2024Towards Trustworthy AI: A Review of Ethical and Robust Large Language Modelscited by 21
  15. 2024Aloe: A Family of Fine-tuned Open Healthcare LLMscited by 20
  16. 2024Has Multimodal Learning Delivered Universal Intelligence in Healthcare? A Comprehensive Surveycited by 20
  17. 2024Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study Over Open-ended Question Answeringcited by 20
  18. 2024Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQAcited by 19
  19. 2025TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Modelscited by 19
  20. 2025Automatic Evaluation of Healthcare LLMs Beyond Question-Answeringcited by 19
  21. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 18
  22. 2024Healthcare Copilot: Eliciting the Power of General LLMs for Medical Consultationcited by 17
  23. 2024A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluationscited by 16
  24. 2024A Survey on Trustworthiness in Foundation Models for Medical Image Analysiscited by 16
  25. 2024On the Vulnerability of LLM/VLM-Controlled Roboticscited by 15
  26. 2024To Generate or to Retrieve? On the Effectiveness of Artificial Contexts for Medical Open-Domain Question Answeringcited by 15
  27. 2024Towards Rationality in Language and Multimodal Agents: A Surveycited by 15
  28. 2024Exploring LLM Multi-Agents for ICD Codingcited by 14
  29. 2025A Systematic Evaluation of LLM Strategies for Mental Health Text Analysis: Fine-tuning vs. Prompt Engineering vs. RAGcited by 14
  30. 2025When Models Lie, We Learn: Multilingual Span-Level Hallucination Detection with PsiloQAcited by 13
  31. 2024Large Language Models in the Clinic: A Comprehensive Benchmarkcited by 12
  32. 2025Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metricscited by 12
  33. 2023Bias in Emotion Recognition with ChatGPTcited by 11
  34. 2024BianCang: A Traditional Chinese Medicine Large Language Modelcited by 11
  35. 2025Self-supervised Quantized Representation for Seamlessly Integrating Knowledge Graphs with Large Language Modelscited by 11
  36. 2026PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebracited by 11
  37. 2024Editing Factual Knowledge and Explanatory Ability of Medical Large Language Modelscited by 10
  38. 2024Hummer: Towards Limited Competitive Preference Datasetcited by 10

50 most cited of 162 All 162 (JSON)