mlsanityData to 14 Oct 2026

The Shaky Foundations of Clinical Foundation Models: A Survey of Large Language Models and Foundation Models for EMRs

arXiv 2303.12961 ↗First posted 22 Mar 2023cs.LG cs.AIcited by 29

Not on the map. Why?

Abstract (arXiv)

The successes of foundation models such as ChatGPT and AlphaFold have spurred significant interest in building similar models for electronic medical records (EMRs) to improve patient care and hospital operations. However, recent hype has obscured critical gaps in our understanding of these models' capabilities. We review over 80 foundation models trained on non-imaging EMR data (i.e. clinical text and/or structured data) and create a taxonomy delineating their architectures, training data, and potential use cases. We find that most models are trained on small, narrowly-scoped clinical datasets (e.g. MIMIC-III) or broad, public biomedical corpora (e.g. PubMed) and are evaluated on tasks that do not provide meaningful insights on their usefulness to health systems. In light of these findings, we propose an improved evaluation framework for measuring the benefits of clinical foundation models that is more closely grounded to metrics that matter in healthcare.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 42 of its 97 references that are in the map. First posted on arXiv 22 Mar 2023.

Builds on

42 of 97 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  4. 2022Emergent Abilities of Large Language Modelscited by 2,512
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  6. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  7. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  8. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  9. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 590
  10. 2022Memory-Based Model Editing at Scalecited by 447
  11. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 407
  12. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 373
30 more
  1. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 339
  2. 2018Scalable and accurate deep learning for electronic health recordscited by 280
  3. 2023The Impact of AI on Developer Productivity: Evidence from GitHub Copilotcited by 274
  4. 2015Doctor AI: Predicting Clinical Events via Recurrent Neural Networkscited by 208
  5. 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 207
  6. 2022Can large language models reason about medical questions?cited by 194
  7. 2022Do Users Write More Insecure Code with AI Assistants?cited by 192
  8. 2021Towards artificial general intelligence via a multimodal foundation modelcited by 112
  9. 2022ChatGPT Makes Medicine Easy to Swallow: An Exploratory Case Study on Simplified Radiology Reportscited by 101
  10. 2020The Cost of Training NLP Models: A Concise Overviewcited by 98
  11. 2023Do We Still Need Clinical Language Models?cited by 98
  12. 2020Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Datacited by 87
  13. 2021Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarizationcited by 76
  14. 2021CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction taskscited by 76
  15. 2022Integrated multimodal artificial intelligence framework for healthcare applicationscited by 71
  16. 2021Hi-BEHRT: Hierarchical Transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health recordscited by 54
  17. 2020Deep Representation Learning of Patient Data from Electronic Health Records (EHR): A Systematic Reviewcited by 47
  18. 2021GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domaincited by 36
  19. 2022BigBIO: A Framework for Data-Centric Biomedical Natural Language Processingcited by 36
  20. 2019Clinical XLNet: Modeling Sequential Clinical Notes and Predicting Prolonged Mechanical Ventilationcited by 30
  21. 2021Unifying Heterogeneous Electronic Health Records Systems via Text-Based Code Embeddingcited by 24
  22. 2020Clinical Reading Comprehension: A Thorough Analysis of the emrQA Datasetcited by 23
  23. 2022Artificial Intelligence-Based Methods for Fusion of Electronic Health Records and Imaging Datacited by 18
  24. 2022Quality Metrics in Recommender Systems: Do We Calculate Metrics Consistently?cited by 17
  25. 2022Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionalscited by 16
  26. 2021A Scoping Review of Publicly Available Language Tasks in Clinical Natural Language Processingcited by 10
  27. 2022On the Importance of Clinical Notes in Multi-modal Learning for EHR Datacited by 10
  28. 2022Developing a general-purpose clinical language inference model from a large corpus of clinical notescited by 8
  29. 2023Clinical Decision Transformer: Intended Treatment Recommendation through Goal Promptingcited by 8
  30. 2022HAPI: A Large-scale Longitudinal Dataset of Commercial ML API Predictionscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 14; 2025: 8; 2026: 2.

29 papers.

Sort:

By area:

  1. 2023EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Modelscited by 105
  2. 2023Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare deliverycited by 30
  3. 2024Data-Centric Foundation Models in Computational Healthcare: A Surveycited by 27
  4. 2023Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicinecited by 24
  5. 2025MedAgentGym: A Scalable Agentic Training Environment for Code-Centric Reasoning in Biomedical Data Sciencecited by 23
  6. 2023Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Modelscited by 20
  7. 2025Training LLMs for EHR-Based Reasoning Tasks via Reinforcement Learningcited by 14
  8. 2024Educational Personalized Learning Path Planning with Large Language Modelscited by 5
  9. 2024Identifying Reasons for Contraceptive Switching from Real-World Data Using Large Language Modelscited by 4
  10. 2023Cost-Sensitive Best Subset Selection for Logistic Regression: A Mixed-Integer Conic Optimization Perspectivecited by 3
  11. 2025End-to-End Chart Summarization via Visual Chain-of-Thought in Vision-Language Modelscited by 3
  12. 2025Interpretable and Robust Dialogue State Tracking via Natural Language Summarization with LLMscited by 3
17 more
  1. 2024Data augmentation method for modeling health records with applications to clopidogrel treatment failure detectioncited by 2
  2. 2024Embedding-based Multimodal Learning on Pan-Squamous Cell Carcinomas for Improved Survival Outcomescited by 2
  3. 2024Large Vision-Language Models for Remote Sensing Visual Question Answeringcited by 2
  4. 2024Fault Diagnosis in Power Grids with Large Language Modelcited by 1
  5. 2024Enhancing Agricultural Machinery Management through Advanced LLM Integrationcited by 1
  6. 2024Foundation Model or Finetune? Evaluation of few-shot semantic segmentation for river pollutioncited by 1
  7. 2025Generating Medically-Informed Explanations for Depression Detection using LLMscited by 1
  8. 2024Machine Learning Predicts Upper Secondary Education Dropout as Early as the End of Primary Schoolcited by 0
  9. 2024LLMs for Enhanced Agricultural Meteorological Recommendationscited by 0
  10. 2024Enhancing AI-Driven Psychological Consultation: Layered Prompts with Large Language Modelscited by 0
  11. 2024Indication Finding: a novel use case for representation learningcited by 0
  12. 2024Align, Generate, Learn: A Novel Closed-Loop Framework for Cross-Lingual In-Context Learningcited by 0
  13. 2025Enhancing Contrastive Demonstration Selection with Semantic Diversity for Robust In-Context Machine Translationcited by 0
  14. 2025Harnessing Generative LLMs for Enhanced Financial Event Entity Extraction Performancecited by 0
  15. 2025Revolutionizing Radiology Workflow with Factual and Efficient CXR Report Generationcited by 0
  16. 2026Epistemic Compression: The Case for Deliberate Ignorance in High-Stakes AIcited by 0
  17. 2026Preferred, Not Safer: Pairwise Preference Is a Poor Proxy for Clinical Safetycited by 0