mlsanityData to 14 Oct 2026

Foundation models for electronic health records: representation dynamics and transferability

arXiv 2504.10422 ↗First posted 14 Apr 2025cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Foundation models (FMs) trained on electronic health records (EHRs) have shown strong performance on a range of clinical prediction tasks. However, adapting these models to local health systems remains challenging due to limited data availability and resource constraints. In this study, we investigated what these models learn and evaluated the transferability of an FM trained on MIMIC-IV to an institutional EHR dataset at the University of Chicago Medical Center. We assessed their ability to identify outlier patients and examined representation-space patient trajectories in relation to future clinical outcomes. We also evaluated the performance of supervised fine-tuned classifiers on both source and target datasets. Our findings offer insights into the adaptability of FMs across different healthcare systems, highlight considerations for their effective implementation, and provide an empirical analysis of the underlying factors that contribute to their predictive performance.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 19 of its 40 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

19 of 40 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  4. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  5. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 373
  6. 2018Scalable and accurate deep learning for electronic health recordscited by 280
  7. 2019BEHRT: Transformer for Electronic Health Recordscited by 263
  8. 2015Doctor AI: Predicting Clinical Events via Recurrent Neural Networkscited by 208
  9. 2023EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Modelscited by 105
  10. 2015Learning to Diagnose with LSTM Recurrent Neural Networkscited by 99
  11. 2020Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Datacited by 87
  12. 2023MOTOR: A Time-To-Event Foundation Model For Structured Medical Recordscited by 78
7 more
  1. 2024Zero Shot Health Trajectory Prediction Using Transformercited by 77
  2. 2023Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Eventscited by 46
  3. 2024Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRscited by 39
  4. 2024EHRMamba: Towards Generalizable and Scalable Foundation Models for Electronic Health Recordscited by 32
  5. 2023A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Recordscited by 31
  6. 2024ClinicalMamba: A Generative Clinical Language Model on Longitudinal Clinical Notescited by 22
  7. 2025Foundation Model of Electronic Medical Records for Adaptive Risk Estimationcited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 10.

11 papers.

  1. 2026One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Modelscited by 5
  2. 2025Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation modelscited by 3
  3. 2026Learning Clinical Representations Under Systematic Distribution Shiftcited by 2
  4. 2026Discriminative Representation Learning for Clinical Predictioncited by 2
  5. 2026Clin-JEPA: A Multi-Phase Co-Training Framework for Joint-Embedding Predictive Pretraining on EHR Patient Trajectoriescited by 2
  6. 2026Uncertainty-Aware Foundation Models for Clinical Datacited by 1
  7. 2026Event Fields: Learning Latent Event Structure for Waveform Foundation Modelscited by 0
  8. 2026WISTERIA: Learning Clinical Representations from Noisy Supervision via Multi-View Consistency in Electronic Health Recordscited by 0
  9. 2026AURORA: Contextual Orthogonalization for Geometric Representation Learning in Healthcare Foundation Modelscited by 0
  10. 2026MorphologyFM: A Foundation Model for Morphology-Aware Representation Learning from ECG and Pulse Oximetry Waveformscited by 0
  11. 2026Federated generative event models for tokenized electronic health recordscited by 0