mlsanityData to 14 Oct 2026

Clinical Risk Prediction Using Language Models: Benefits And Considerations

arXiv 2312.03742 ↗First posted 29 Nov 2023cs.CL cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

The utilization of Electronic Health Records (EHRs) for clinical risk prediction is on the rise. However, strict privacy regulations limit access to comprehensive health records, making it challenging to apply standard machine learning algorithms in practical real-world scenarios. Previous research has addressed this data limitation by incorporating medical ontologies and employing transfer learning methods. In this study, we investigate the potential of leveraging language models (LMs) as a means to incorporate supplementary domain knowledge for improving the performance of various EHR-based risk prediction tasks. Unlike applying LMs to unstructured EHR data such as clinical notes, this study focuses on using textual descriptions within structured EHR to make predictions exclusively based on that information. We extensively compare against previous approaches across various data types and sizes. We find that employing LMs to represent structured EHRs, such as diagnostic histories, leads to improved or at least comparable performance in diverse risk prediction tasks. Furthermore, LM-based approaches offer numerous advantages, including few-shot learning, the capability to handle previously unseen medical concepts, and adaptability to various medical vocabularies. Nevertheless, we underscore, through various experiments, the importance of being cautious when employing such models, as concerns regarding the reliability of LMs persist.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 21 of its 41 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

21 of 41 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  6. 2023A Comprehensive Overview of Large Language Modelscited by 648
  7. 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 493
  8. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 373
  9. 2016RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanismcited by 349
  10. 2019BEHRT: Transformer for Electronic Health Recordscited by 263
  11. 2015Doctor AI: Predicting Clinical Events via Recurrent Neural Networkscited by 208
  12. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 169
9 more
  1. 2016GRAM: Graph-based Attention Model for Healthcare Representation Learningcited by 156
  2. 2017Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysiscited by 148
  3. 2019Pre-training of Graph Augmented Transformers for Medication Recommendationcited by 127
  4. 2021CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction taskscited by 76
  5. 2021Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mappingcited by 48
  6. 2018Patient2Vec: A Personalized Interpretable Deep Representation of the Longitudinal Electronic Health Recordcited by 30
  7. 2019Benchmarking Deep Learning Architectures for Predicting Readmission to the ICU and Describing Patients-at-Riskcited by 17
  8. 2022Self-supervised Pre-training for Semantic Segmentation in an Indoor Scenecited by 1
  9. 2022GenSyn: A Multi-stage Framework for Generating Synthetic Microdata using Macro Data Sourcescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 4.

6 papers.

  1. 2025Large Language Models are Powerful Electronic Health Record Encoderscited by 26
  2. 2024DeLLiriuM: A large language model for delirium prediction in the ICU using structured EHRcited by 4
  3. 2026Retrieval-aligned Tabular Foundation Models Enable Robust Clinical Risk Prediction in Electronic Health Records Under Real-world Constraintscited by 4
  4. 2026AgentRx: A Benchmark Study of LLM Agents for Multimodal Clinical Prediction Taskscited by 3
  5. 2026PaReGTA: A Temporally Aware LLM-Based Patient Representation Framework for EHR Analyticscited by 1
  6. 2026A General Multimodal Probing Framework for Data Fusion and Prediction with Frozen Large Language Models: Evidence from Multimodal EHR Datacited by 0