mlsanityData to 14 Oct 2026

Scaling Electronic Health Record Foundation Models for Population Health Management

arXiv 2506.00209 ↗First posted 30 May 2025cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Population health management requires scalable methods to identify individuals at risk of chronic diseases such as cardiovascular conditions and cancer, yet existing approaches rely on fragmented data and resource-intensive screening. We present Scaling Electronic Health Record Foundation Models for Population Health Management, an Electronic Health Record Foundation Model that performs large-scale chronic disease prediction using cross-site longitudinal medical records. We pretrain Scaling Electronic Health Record Foundation Models for Population Health Management on billions of medical events from over 5 million patients across Taiwan and the United States, leveraging a unified code alignment framework to address cross-system heterogeneity, and characterize its scaling behavior via IsoFLOP analysis, training compute-optimal models up to 2.4B parameters. Across 11 chronic disease prediction tasks, Scaling Electronic Health Record Foundation Models for Population Health Management demonstrates strong scaling and generalization, outperforming tree-based models and both general and biomedical language models, achieving over 40% and 70% sensitivity at 99% specificity in U.S. and Taiwan cohorts, respectively. On the EHRShot benchmark, Scaling Electronic Health Record Foundation Models for Population Health Management surpasses prior EHR foundation models trained on in-site data despite substantial distribution shifts, highlighting strong few-shot generalization. Finally, we show that aligned cross-system data provides more effective pretraining signal than duplicating single-site data under data-limited settings, underscoring the importance of alignment for scalable healthcare modeling. Our analysis demonstrates the robustness of EHR-FM in various patient distributions and the benefits of operating in the ICD code space. The code will be open-sourced.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 74 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

24 of 74 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  7. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 372
  8. 2018Scalable and accurate deep learning for electronic health recordscited by 280
  9. 2020Self-Alignment Pretraining for Biomedical Entity Representationscited by 237
  10. 2020Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Taskscited by 138
  11. 2023EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Modelscited by 105
  12. 2023MOTOR: A Time-To-Event Foundation Model For Structured Medical Recordscited by 78
12 more
  1. 2024Zero Shot Health Trajectory Prediction Using Transformercited by 77
  2. 2020Biomedical Entity Representations with Synonym Marginalizationcited by 61
  3. 2020CODER: Knowledge infused cross-lingual medical term embedding for term normalizationcited by 55
  4. 2023Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Eventscited by 46
  5. 2020Deep Representation Learning of Electronic Health Records to Unlock Patient Stratification at Scalecited by 39
  6. 2023Using Sequences of Life-events to Predict Human Livescited by 27
  7. 2023A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?cited by 24
  8. 2024Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generationcited by 23
  9. 2024Linq-Embed-Mistral Technical Reportcited by 20
  10. 2025Exploring Scaling Laws for EHR Foundation Modelscited by 12
  11. 2022Foresight -- Generative Pretrained Transformer (GPT) for Modelling of Patient Timelines using EHRscited by 11
  12. 2024ED-Copilot: Reduce Emergency Department Wait Time with Language Model Diagnostic Assistancecited by 9

Built on by

Not cited yet.