# Generative Medical Event Models Improve with Scale

arXiv: https://arxiv.org/abs/2508.12104 · HTML: https://mlsanity.com/paper/2508.12104/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2025.

## Built on by (most cited first, 26 of 26)

- [Beyond Generative AI: World Models for Clinical Prediction, Counterfactuals, and Planning](https://mlsanity.com/paper/2511.16333/) (2025)
- [A multimodal and temporal foundation model for virtual patient representations at healthcare system scale](https://mlsanity.com/paper/2604.18570/) (2026)
- [Cross-Representation Benchmarking in Time-Series Electronic Health Records for Clinical Outcome Prediction](https://mlsanity.com/paper/2510.09159/) (2025)
- [Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policies](https://mlsanity.com/paper/2606.16721/) (2026)
- [Medical world models in healthcare: foundations, applications, and challenges for trustworthy clinical translation](https://mlsanity.com/paper/2607.25242/) (2026)
- [Rethinking Tokenization for Clinical Time Series: When Less is More](https://mlsanity.com/paper/2512.05217/) (2025)
- [Scaling Recurrence-aware Foundation Models for Clinical Records via Next-Visit Prediction](https://mlsanity.com/paper/2603.24562/) (2026)
- [Dementia-R1: Reinforced Pretraining and Reasoning from Unstructured Clinical Notes for Real-World Dementia Prognosis](https://mlsanity.com/paper/2601.03018/) (2026)
- [LLMs can construct powerful representations and streamline sample-efficient supervised learning](https://mlsanity.com/paper/2603.11679/) (2026)
- [Integrating Genomics into Multimodal EHR Foundation Models](https://mlsanity.com/paper/2510.23639/) (2025)
- [Vital Trace: Protocol-Constrained Patient-State Reasoning for Longitudinal Clinical Trajectories](https://mlsanity.com/paper/2602.12833/) (2026)
- [From Statistical Fidelity to Clinical Consistency: Scalable Generation and Auditing of Synthetic Patient Trajectories](https://mlsanity.com/paper/2603.06720/) (2026)
- [Foundation Models to Unlock Real-World Evidence from Nationwide Medical Claims](https://mlsanity.com/paper/2605.02740/) (2026)
- [A Systematic Review on the Generative AI Applications in Human Medical Genomics](https://mlsanity.com/paper/2508.20275/) (2025)
- [EveryQuery: A Promptable Foundation Model for Clinical Prediction Tasks over Electronic Health Records](https://mlsanity.com/paper/2603.07900/) (2026)
- [A Nationwide Japanese Medical Claims Foundation Model: Balancing Model Scaling and Task-Specific Computational Efficiency](https://mlsanity.com/paper/2604.22348/) (2026)
- [EHR-RAGp: Retrieval-Augmented Prototype-Guided Foundation Model for Electronic Health Records](https://mlsanity.com/paper/2605.12335/) (2026)
- [Conditional Attribute Estimation with Autoregressive Sequence Models](https://mlsanity.com/paper/2605.14004/) (2026)
- [TEDDY: A Pediatric Foundation Model for Risk Forewarning from ICD-Coded Diagnostic Histories](https://mlsanity.com/paper/2607.14191/) (2026)
- [Autoregressive EHR Foundation Models with Multimodal Inputs](https://mlsanity.com/paper/2607.22264/) (2026)
- [Federated generative event models for tokenized electronic health records](https://mlsanity.com/paper/2608.02939/) (2026)
- [Position: Medical AI Neglects Real Treatment Outcomes](https://mlsanity.com/paper/2608.14598/) (2026)
- [Foresight-England: Development of a National-Scale Generative AI Model of Electronic Health Records for Medical Event Prediction across the COVID-19 Pandemic](https://mlsanity.com/paper/2608.16273/) (2026)
- [A Multimodal Foundation Model for Longitudinal Patient Representation and Scalable Insight Generation in Oncology](https://mlsanity.com/paper/2608.24688/) (2026)
- [Reinforcement Learning over Patient Trajectories for Clinical Reasoning in EHR Foundation Models](https://mlsanity.com/paper/2609.12277/) (2026)
- [Sequence Composition in EHR Foundation Model Pretraining](https://mlsanity.com/paper/2610.16878/) (2026)

## Abstract (arXiv)

Realizing personalized medicine at scale calls for methods that distill insights from longitudinal patient journeys, which can be viewed as a sequence of medical events. Foundation models pretrained on large-scale medical event data represent a promising direction for scaling real-world evidence generation and generalizing to diverse downstream tasks. Using Epic Cosmos, a dataset with medical events from de-identified longitudinal health records for 16.3 billion encounters over 300 million unique patient records from 310 health systems, we introduce the Curiosity models, a family of decoder-only transformer models pretrained on 118 million patients representing 115 billion discrete medical events (151 billion tokens). We present the largest scaling-law study of medical event data, establishing a methodology for pretraining and revealing power-law scaling relationships for compute, tokens, and model size. Consequently, we pretrained a series of compute-optimal models with up to 1 billion parameters. Conditioned on a patient's real-world history, Curiosity autoregressively predicts the next medical event to simulate patient health timelines. We studied 78 real-world tasks, including diagnosis prediction, disease prognosis, and healthcare operations. Remarkably for a foundation model with generic pretraining and simulation-based inference, Curiosity generally outperformed or matched task-specific supervised models on these tasks, without requiring task-specific fine-tuning or few-shot examples. Curiosity's predictive power consistently improves as the model and pretraining scale. Our results show that Curiosity, a generative medical event foundation model, can effectively capture complex clinical dynamics, providing an extensible and generalizable framework to support clinical decision-making, streamline healthcare operations, and improve patient outcomes.