# Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Events

arXiv: https://arxiv.org/abs/2306.11547 · HTML: https://mlsanity.com/paper/2306.11547/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 20 Jun 2023.

## Builds on (most cited first, 28 of 28)

- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction](https://mlsanity.com/paper/2005.12833/) (2020)
- [Multitask learning and benchmarking with clinical time series data](https://mlsanity.com/paper/1703.07771/) (2017)
- [BEHRT: Transformer for Electronic Health Records](https://mlsanity.com/paper/1907.09538/) (2019)
- [Transformer Hawkes Process](https://mlsanity.com/paper/2002.09291/) (2020)
- [Doctor AI: Predicting Clinical Events via Recurrent Neural Networks](https://mlsanity.com/paper/1511.05942/) (2015)
- [Are Large Pre-Trained Language Models Leaking Your Personal Information?](https://mlsanity.com/paper/2205.12628/) (2022)
- [Intensity-Free Learning of Temporal Point Processes](https://mlsanity.com/paper/1909.12127/) (2019)
- [MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-III](https://mlsanity.com/paper/1907.08322/) (2019)
- [Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?](https://mlsanity.com/paper/2104.07762/) (2021)
- [Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Data](https://mlsanity.com/paper/2001.05295/) (2020)
- [Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings](https://mlsanity.com/paper/2003.11515/) (2020)
- [CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks](https://mlsanity.com/paper/2111.08585/) (2021)
- [Hi-BEHRT: Hierarchical Transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health records](https://mlsanity.com/paper/2106.11360/) (2021)
- [Neural Temporal Point Processes For Modelling Electronic Health Records](https://mlsanity.com/paper/2007.13794/) (2020)
- [MedGPT: Medical Concept Prediction from Clinical Narratives](https://mlsanity.com/paper/2107.03134/) (2021)
- [Deep Contextual Clinical Prediction with Reverse Distillation](https://mlsanity.com/paper/2007.05611/) (2020)
- [Is Fairness Only Metric Deep? Evaluating and Addressing Subgroup Gaps in Deep Metric Learning](https://mlsanity.com/paper/2203.12748/) (2022)
- [Structure Inducing Pre-Training](https://mlsanity.com/paper/2103.10334/) (2021)
- [Targeted-BEHRT: Deep learning for observational causal inference on longitudinal electronic health records](https://mlsanity.com/paper/2202.03487/) (2022)
- [Unsupervised pre-training of graph transformers on patient population graphs](https://mlsanity.com/paper/2207.10603/) (2022)
- [Clinical Decision Transformer: Intended Treatment Recommendation through Goal Prompting](https://mlsanity.com/paper/2302.00612/) (2023)
- [Leveraging Time Irreversibility with Order-Contrastive Pre-training](https://mlsanity.com/paper/2111.02599/) (2021)
- [TemporAI: Facilitating Machine Learning Innovation in Time Domain Tasks for Medicine](https://mlsanity.com/paper/2301.12260/) (2023)
- [Explainable Health Risk Predictor with Transformer-based Medicare Claim Encoder](https://mlsanity.com/paper/2105.09428/) (2021)
- [Generalized and Transferable Patient Language Representation for Phenotyping with Limited Data](https://mlsanity.com/paper/2103.00482/) (2021)

## Built on by (most cited first, 30 of 46)

- [Zero Shot Health Trajectory Prediction Using Transformer](https://mlsanity.com/paper/2407.21124/) (2024)
- [Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRs](https://mlsanity.com/paper/2412.16178/) (2024)
- [Foundation Model of Electronic Medical Records for Adaptive Risk Estimation](https://mlsanity.com/paper/2502.06124/) (2025)
- [Event-Based Contrastive Learning for Medical Time Series](https://mlsanity.com/paper/2312.10308/) (2023)
- [HoTPP Benchmark: Are We Good at the Long Horizon Events Forecasting?](https://mlsanity.com/paper/2406.14341/) (2024)
- [A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMs](https://mlsanity.com/paper/2504.14657/) (2025)
- [Exploring Scaling Laws for EHR Foundation Models](https://mlsanity.com/paper/2505.22964/) (2025)
- [Foundation models for electronic health records: representation dynamics and transferability](https://mlsanity.com/paper/2504.10422/) (2025)
- [ACES: Automatic Cohort Extraction System for Event-Stream Datasets](https://mlsanity.com/paper/2406.19653/) (2024)
- [Multimodal Banking Dataset: Understanding Client Needs through Event Sequences](https://mlsanity.com/paper/2409.17587/) (2024)
- [A Collection of Innovations in Medical AI for patient records in 2024](https://mlsanity.com/paper/2503.05768/) (2025)
- [Detecting the Future: All-at-Once Event Sequence Forecasting with Horizon Matching](https://mlsanity.com/paper/2408.13131/) (2024)
- [FoMoH: A clinically meaningful foundation model evaluation for structured electronic health records](https://mlsanity.com/paper/2505.16941/) (2025)
- [Generalized Prompt Tuning: Adapting Frozen Univariate Time Series Foundation Models for Multivariate Healthcare Time Series](https://mlsanity.com/paper/2411.12824/) (2024)
- [An Investigation of Memorization Risk in Healthcare Foundation Models](https://mlsanity.com/paper/2510.12950/) (2025)
- [One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models](https://mlsanity.com/paper/2602.00541/) (2026)
- [meds_reader: A fast and efficient EHR processing library](https://mlsanity.com/paper/2409.09095/) (2024)
- [CEHR-XGPT: A Scalable Multi-Task Foundation Model for Electronic Health Records](https://mlsanity.com/paper/2509.03643/) (2025)
- [Recent Advances, Applications, and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2023 Symposium](https://mlsanity.com/paper/2403.01628/) (2024)
- [EHR2Path: Comprehensive Pathway-Level Modeling of Longitudinal Patient Trajectories from Multimodal Electronic Health Records](https://mlsanity.com/paper/2506.04831/) (2025)
- [Foundation Models for Clinical Records at Health System Scale](https://mlsanity.com/paper/2507.00574/) (2025)
- [Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation models](https://mlsanity.com/paper/2507.22798/) (2025)
- [HT-Transformer: Event Sequences Classification by Accumulating Prefix Information with History Tokens](https://mlsanity.com/paper/2508.01474/) (2025)
- [Count-Based Approaches Remain Strong: A Benchmark Against Transformer and LLM Pipelines on Structured EHR](https://mlsanity.com/paper/2511.00782/) (2025)
- [The Taxonomies, Training, and Applications of Event Stream Modelling for Electronic Health Records](https://mlsanity.com/paper/2603.14003/) (2026)
- [Scaling Recurrence-aware Foundation Models for Clinical Records via Next-Visit Prediction](https://mlsanity.com/paper/2603.24562/) (2026)
- [Efficient Generative Prediction for EHR Foundation Models: The SCOPE and REACH Estimators](https://mlsanity.com/paper/2602.03730/) (2026)
- [Learning Clinical Representations Under Systematic Distribution Shift](https://mlsanity.com/paper/2603.07348/) (2026)
- [Discriminative Representation Learning for Clinical Prediction](https://mlsanity.com/paper/2603.20921/) (2026)
- [ESQA: Event Sequences Question Answering](https://mlsanity.com/paper/2407.12833/) (2024)

## Abstract (arXiv)

Generative, pre-trained transformers (GPTs, a.k.a. "Foundation Models") have reshaped natural language processing (NLP) through their versatility in diverse downstream tasks. However, their potential extends far beyond NLP. This paper provides a software utility to help realize this potential, extending the applicability of GPTs to continuous-time sequences of complex events with internal dependencies, such as medical record datasets. Despite their potential, the adoption of foundation models in these domains has been hampered by the lack of suitable tools for model construction and evaluation. To bridge this gap, we introduce Event Stream GPT (ESGPT), an open-source library designed to streamline the end-to-end process for building GPTs for continuous-time event sequences. ESGPT allows users to (1) build flexible, foundation-model scale input datasets by specifying only a minimal configuration file, (2) leverage a Hugging Face compatible modeling API for GPTs over this modality that incorporates intra-event causal dependency structures and autoregressive generation capabilities, and (3) evaluate models via standardized processes that can assess few and even zero-shot performance of pre-trained models on user-specified fine-tuning tasks.