mlsanityData to 14 Oct 2026

CEHR-XGPT: A Scalable Multi-Task Foundation Model for Electronic Health Records

arXiv 2509.03643 ↗First posted 3 Sep 2025cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Electronic Health Records (EHRs) provide a rich, longitudinal view of patient health and hold significant potential for advancing clinical decision support, risk prediction, and data-driven healthcare research. However, most artificial intelligence (AI) models for EHRs are designed for narrow, single-purpose tasks, limiting their generalizability and utility in real-world settings. Here, we present CEHR-XGPT, a general-purpose foundation model for EHR data that unifies three essential capabilities - feature representation, zero-shot prediction, and synthetic data generation - within a single architecture. To support temporal reasoning over clinical sequences, CEHR-XGPT incorporates a novel time-token-based learning framework that explicitly encodes patients' dynamic timelines into the model structure. CEHR-XGPT demonstrates strong performance across all three tasks and generalizes effectively to external datasets through vocabulary expansion and fine-tuning. Its versatility enables rapid model development, cohort discovery, and patient outcome forecasting without the need for task-specific retraining.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 23 of its 42 references that are in the map. First posted on arXiv 3 Sep 2025.

Builds on

23 of 42 references.

References · 23

  1. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  2. 2019BEHRT: Transformer for Electronic Health Recordscited by 263
  3. 2017Generating Multi-label Discrete Patient Records using Generative Adversarial Networkscited by 197
  4. 2022Transformer Language Models without Positional Encodings Still Learn Positional Informationcited by 150
  5. 2020Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Datacited by 87
  6. 2023MOTOR: A Time-To-Event Foundation Model For Structured Medical Recordscited by 78
  7. 2024Zero Shot Health Trajectory Prediction Using Transformercited by 77
  8. 2021CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction taskscited by 76
  9. 2021Hi-BEHRT: Hierarchical Transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health recordscited by 54
  10. 2023Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Eventscited by 46
  11. 2022A Multifaceted Benchmarking of Synthetic Electronic Health Record Generation Modelscited by 43
  12. 2024Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRscited by 39
11 more
  1. 2020CorGAN: Correlation-Capturing Convolutional Generative Adversarial Networks for Generating Synthetic Healthcare Recordscited by 37
  2. 2017Analyzing the Robustness of Nearest Neighbors to Adversarial Examplescited by 36
  3. 2022GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learningcited by 36
  4. 2024EHRMamba: Towards Generalizable and Scalable Foundation Models for Electronic Health Recordscited by 32
  5. 2024CEHR-GPT: Generating Electronic Health Records with Chronological Patient Timelinescited by 22
  6. 2020Generating Electronic Health Records with Multiple Data Types and Constraintscited by 21
  7. 2024CORE-BEHRT: A Carefully Optimized and Rigorously Evaluated BEHRTcited by 20
  8. 2017Time-Dependent Representation for Neural Event Sequence Predictioncited by 11
  9. 2022Foresight -- Generative Pretrained Transformer (GPT) for Modelling of Patient Timelines using EHRscited by 11
  10. 2019CONAN: Complementary Pattern Augmentation for Rare Disease Detectioncited by 9
  11. 2022Targeted-BEHRT: Deep learning for observational causal inference on longitudinal electronic health recordscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026Retrieval-aligned Tabular Foundation Models Enable Robust Clinical Risk Prediction in Electronic Health Records Under Real-world Constraintscited by 4
  2. 2026Scaling Recurrence-aware Foundation Models for Clinical Records via Next-Visit Predictioncited by 3
  3. 2026PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Modelscited by 2
  4. 2025Toward Scalable Early Cancer Detection: Evaluating EHR-Based Predictive Models Against Traditional Screening Criteriacited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Prediction: Longitudinal Reasoning in EHR-Integrated Clinical AI ↗
  2. 2026ChatHealthAI: Aligning Electronic Health Record Representations with Large Language Models for Grounded Clinical Reasoning
  3. 2025Building the EHR Foundation Model via Next Event Prediction
  4. 2026EHRWorld: A Patient-Centric Medical World Model for Long-Horizon Clinical Trajectories
  5. 2026Reinforcement Learning over Patient Trajectories for Clinical Reasoning in EHR Foundation Models
  6. 2025EHR-R1: A Reasoning-Enhanced Foundational Language Model for Electronic Health Record Analysis
  7. 2026EHR-RAG: Bridging Long-Horizon Structured Electronic Health Records and Large Language Models via Enhanced Retrieval-Augmented Generation
  8. 2026COTCAgent: Preventive Consultation via Probabilistic Chain-of-Thought Completion