# Foundation Model of Electronic Medical Records for Adaptive Risk Estimation

arXiv: https://arxiv.org/abs/2502.06124 · HTML: https://mlsanity.com/paper/2502.06124/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 54 references that are in the map.
- First posted on arXiv 10 Feb 2025.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [BEHRT: Transformer for Electronic Health Records](https://mlsanity.com/paper/1907.09538/) (2019)
- [Zero Shot Health Trajectory Prediction Using Transformer](https://mlsanity.com/paper/2407.21124/) (2024)
- [Multi-layer Representation Learning for Medical Concepts](https://mlsanity.com/paper/1602.05568/) (2016)
- [Large language models surpass human experts in predicting neuroscience results](https://mlsanity.com/paper/2403.03230/) (2024)
- [Hi-BEHRT: Hierarchical Transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health records](https://mlsanity.com/paper/2106.11360/) (2021)
- [Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Events](https://mlsanity.com/paper/2306.11547/) (2023)
- [Benchmarking emergency department triage prediction models with machine learning and large public electronic health records](https://mlsanity.com/paper/2111.11017/) (2021)
- [Event-Based Contrastive Learning for Medical Time Series](https://mlsanity.com/paper/2312.10308/) (2023)
- [MEDS-Tab: Automated tabularization and baseline methods for MEDS datasets](https://mlsanity.com/paper/2411.00200/) (2024)

## Built on by (most cited first, 19 of 19)

- [EHR-R1: A Reasoning-Enhanced Foundational Language Model for Electronic Health Record Analysis](https://mlsanity.com/paper/2510.25628/) (2025)
- [Foundation models for electronic health records: representation dynamics and transferability](https://mlsanity.com/paper/2504.10422/) (2025)
- [Evolving Interactive Diagnostic Agents in a Virtual Clinical Environment](https://mlsanity.com/paper/2510.24654/) (2025)
- [Tokenization Tradeoffs in Structured EHR Foundation Models](https://mlsanity.com/paper/2603.15644/) (2026)
- [Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation models](https://mlsanity.com/paper/2507.22798/) (2025)
- [The Taxonomies, Training, and Applications of Event Stream Modelling for Electronic Health Records](https://mlsanity.com/paper/2603.14003/) (2026)
- [Efficient Generative Prediction for EHR Foundation Models: The SCOPE and REACH Estimators](https://mlsanity.com/paper/2602.03730/) (2026)
- [EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twins](https://mlsanity.com/paper/2607.08793/) (2026)
- [Integrating Genomics into Multimodal EHR Foundation Models](https://mlsanity.com/paper/2510.23639/) (2025)
- [Toward Scalable Early Cancer Detection: Evaluating EHR-Based Predictive Models Against Traditional Screening Criteria](https://mlsanity.com/paper/2511.11293/) (2025)
- [Clinical Data Goes MEDS? Let's OWL make sense of it](https://mlsanity.com/paper/2601.04164/) (2026)
- [EveryQuery: A Promptable Foundation Model for Clinical Prediction Tasks over Electronic Health Records](https://mlsanity.com/paper/2603.07900/) (2026)
- [EHR-RAGp: Retrieval-Augmented Prototype-Guided Foundation Model for Electronic Health Records](https://mlsanity.com/paper/2605.12335/) (2026)
- [X-FEMR: A Token-level Explainable Approach for Electronic Health Records Foundation Models using Transformer-based Models](https://mlsanity.com/paper/2607.06163/) (2026)
- [Pretraining EHR Foundation Models with Patient-Aware Sampling](https://mlsanity.com/paper/2607.22114/) (2026)
- [Federated generative event models for tokenized electronic health records](https://mlsanity.com/paper/2608.02939/) (2026)
- [Reinforcement Learning over Patient Trajectories for Clinical Reasoning in EHR Foundation Models](https://mlsanity.com/paper/2609.12277/) (2026)
- [Hidden Data Challenges in Clinical Machine Learning](https://mlsanity.com/paper/2610.13350/) (2026)
- [Sequence Composition in EHR Foundation Model Pretraining](https://mlsanity.com/paper/2610.16878/) (2026)

## Abstract (arXiv)

Hospitals struggle to predict critical outcomes. Traditional early warning systems, like NEWS and MEWS, rely on static variables and fixed thresholds, limiting their adaptability, accuracy, and personalization. We previously developed the Enhanced Transformer for Health Outcome Simulation (ETHOS), an AI model that tokenizes patient health timelines (PHTs) from EHRs and uses transformer-based architectures to predict future PHTs. ETHOS is a versatile framework for developing a wide range of applications. In this work, we develop the Adaptive Risk Estimation System (ARES) that leverages ETHOS to compute dynamic, personalized risk probabilities for clinician-defined critical events. ARES also features a personalized explainability module that highlights key clinical factors influencing risk estimates. We evaluated ARES using the MIMIC-IV v2.2 dataset together with its Emergency Department (ED) extension and benchmarked performance against both classical early warning systems and contemporary machine learning models. The entire dataset was tokenized resulting in 285,622 PHTs, comprising over 360 million tokens. ETHOS outperformed benchmark models in predicting hospital admissions, ICU admissions, and prolonged stays, achieving superior AUC scores. Its risk estimates were robust across demographic subgroups, with calibration curves confirming model reliability. The explainability module provided valuable insights into patient-specific risk factors. ARES, powered by ETHOS, advances predictive healthcare AI by delivering dynamic, real-time, personalized risk estimation with patient-specific explainability. Although our results are promising, the clinical impact remains uncertain. Demonstrating ARES's true utility in real-world settings will be the focus of our future work. We release the source code to facilitate future research.