mlsanityData to 7 Oct 2026

Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Data

arXiv 2001.05295 ↗First posted 6 Jan 2020cs.CL cs.LG stat.MLcited by 85

Not on the map. Why?

Abstract (arXiv)

Widespread adoption of electronic health records (EHRs) has fueled the development of using machine learning to build prediction models for various clinical outcomes. This process is often constrained by having a relatively small number of patient records for training the model. We demonstrate that using patient representation schemes inspired from techniques in natural language processing can increase the accuracy of clinical prediction models by transferring information learned from the entire patient population to the task of training a specific model, where only a subset of the population is relevant. Such patient representation schemes enable a 3.5% mean improvement in AUROC on five prediction tasks compared to standard baselines, with the average improvement rising to 19% when only a small number of patient records are available for training the clinical prediction model.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 4; 2023: 21; 2024: 13; 2025: 19; 2026: 22.

85 papers.

Sort:

By area:

  1. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  2. 2024The Platonic Representation Hypothesiscited by 362
  3. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  4. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  5. 2023EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Modelscited by 101
  6. 2023MOTOR: A Time-To-Event Foundation Model For Structured Medical Recordscited by 75
  7. 2024Zero Shot Health Trajectory Prediction Using Transformercited by 74
  8. 2023Multimodal LLMs for health grounded in individual-specific datacited by 48
  9. 2023Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Eventscited by 46
  10. 2023Benefits and Harms of Large Language Models in Digital Mental Healthcited by 39
  11. 2024Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRscited by 39
  12. 2022GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learningcited by 35
38 more
  1. 2023CPLLM: Clinical Prediction with Large Language Modelscited by 34
  2. 2024On the Challenges and Opportunities in Generative AIcited by 31
  3. 2023A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Recordscited by 30
  4. 2023The Shaky Foundations of Clinical Foundation Models: A Survey of Large Language Models and Foundation Models for EMRscited by 29
  5. 2023Natural Language Processing in Electronic Health Records in Relation to Healthcare Decision-making: A Systematic Reviewcited by 25
  6. 2025Large Language Models are Powerful Electronic Health Record Encoderscited by 25
  7. 2021Unifying Heterogeneous Electronic Health Records Systems via Text-Based Code Embeddingcited by 24
  8. 2023Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicinecited by 24
  9. 2024Emergency Department Decision Support using Clinical Pseudo-notescited by 22
  10. 2023Transfer Knowledge from Natural Language to Electrocardiography: Can We Detect Cardiovascular Disease Through Language Models?cited by 21
  11. 2023When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence Systemcited by 13
  12. 2023Event-Based Contrastive Learning for Medical Time Seriescited by 13
  13. 2024RelayAttention for Efficient Large Language Model Serving with Long System Promptscited by 13
  14. 2021A comparison of approaches to improve worst-case predictive model performance over patient subpopulationscited by 12
  15. 2023General-Purpose Retrieval-Enhanced Medical Prediction Model Using Near-Infinite Historycited by 12
  16. 2025ChronoFormer: Time-Aware Transformer Architectures for Structured Clinical Event Modelingcited by 11
  17. 2025Foundation models for electronic health records: representation dynamics and transferabilitycited by 11
  18. 2021Leveraging Time Irreversibility with Order-Contrastive Pre-trainingcited by 7
  19. 2025FoMoH: A clinically meaningful foundation model evaluation for structured electronic health recordscited by 6
  20. 2026EHR-RAG: Bridging Long-Horizon Structured Electronic Health Records and Large Language Models via Enhanced Retrieval-Augmented Generationcited by 6
  21. 2026Tokenization Tradeoffs in Structured EHR Foundation Modelscited by 6
  22. 2023DEPLOYR: A technical framework for deploying custom real-time machine learning models into the electronic medical recordcited by 5
  23. 2024Large Language Multimodal Models for 5-Year Chronic Disease Cohort Prediction Using EHR Datacited by 5
  24. 2024Representation Learning of Structured Data for Medical Foundation Modelscited by 5
  25. 2025Building the EHR Foundation Model via Next Event Predictioncited by 5
  26. 2026One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Modelscited by 5
  27. 2021Explainable Health Risk Predictor with Transformer-based Medicare Claim Encodercited by 4
  28. 2024Towards a Foundation Model for Brain Age Prediction using coVariance Neural Networkscited by 4
  29. 2025CEHR-XGPT: A Scalable Multi-Task Foundation Model for Electronic Health Recordscited by 4
  30. 2025Cross-Representation Benchmarking in Time-Series Electronic Health Records for Clinical Outcome Predictioncited by 4
  31. 2021Generalized and Transferable Patient Language Representation for Phenotyping with Limited Datacited by 3
  32. 2022sEHR-CE: Language modelling of structured EHR data for efficient and generalizable patient cohort expansioncited by 3
  33. 2023Redefining Digital Health Interfaces with Large Language Modelscited by 3
  34. 2023Cost-Sensitive Best Subset Selection for Logistic Regression: A Mixed-Integer Conic Optimization Perspectivecited by 3
  35. 2024Unsupervised Discovery of Clinical Disease Signatures Using Probabilistic Independencecited by 3
  36. 2024Uncertainty Quantification for Clinical Outcome Predictions with (Large) Language Modelscited by 3
  37. 2025Foundation Models for Clinical Records at Health System Scalecited by 3
  38. 2025Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation modelscited by 3

50 most cited of 85 All 85 (JSON)