mlsanityData to 14 Oct 2026

Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Events

arXiv 2306.11547 ↗First posted 20 Jun 2023cs.LGcited by 46

Not on the map. Why?

Abstract (arXiv)

Generative, pre-trained transformers (GPTs, a.k.a. "Foundation Models") have reshaped natural language processing (NLP) through their versatility in diverse downstream tasks. However, their potential extends far beyond NLP. This paper provides a software utility to help realize this potential, extending the applicability of GPTs to continuous-time sequences of complex events with internal dependencies, such as medical record datasets. Despite their potential, the adoption of foundation models in these domains has been hampered by the lack of suitable tools for model construction and evaluation. To bridge this gap, we introduce Event Stream GPT (ESGPT), an open-source library designed to streamline the end-to-end process for building GPTs for continuous-time event sequences. ESGPT allows users to (1) build flexible, foundation-model scale input datasets by specifying only a minimal configuration file, (2) leverage a Hugging Face compatible modeling API for GPTs over this modality that incorporates intra-event causal dependency structures and autoregressive generation capabilities, and (3) evaluate models via standardized processes that can assess few and even zero-shot performance of pre-trained models on user-specified fine-tuning tasks.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). Builds on 28 of its 48 references that are in the map. First posted on arXiv 20 Jun 2023.

Builds on

28 of 48 references.

References · 28

  1. 2018Datasheets for Datasetscited by 1,386
  2. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  3. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  4. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 373
  5. 2017Multitask learning and benchmarking with clinical time series datacited by 290
  6. 2019BEHRT: Transformer for Electronic Health Recordscited by 263
  7. 2020Transformer Hawkes Processcited by 235
  8. 2015Doctor AI: Predicting Clinical Events via Recurrent Neural Networkscited by 208
  9. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 200
  10. 2019Intensity-Free Learning of Temporal Point Processescited by 140
  11. 2019MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-IIIcited by 122
  12. 2021Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?cited by 93
16 more
  1. 2020Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Datacited by 87
  2. 2020Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddingscited by 79
  3. 2021CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction taskscited by 76
  4. 2021Hi-BEHRT: Hierarchical Transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health recordscited by 54
  5. 2020Neural Temporal Point Processes For Modelling Electronic Health Recordscited by 39
  6. 2021MedGPT: Medical Concept Prediction from Clinical Narrativescited by 24
  7. 2020Deep Contextual Clinical Prediction with Reverse Distillationcited by 16
  8. 2022Is Fairness Only Metric Deep? Evaluating and Addressing Subgroup Gaps in Deep Metric Learningcited by 11
  9. 2021Structure Inducing Pre-Trainingcited by 10
  10. 2022Targeted-BEHRT: Deep learning for observational causal inference on longitudinal electronic health recordscited by 9
  11. 2022Unsupervised pre-training of graph transformers on patient population graphscited by 9
  12. 2023Clinical Decision Transformer: Intended Treatment Recommendation through Goal Promptingcited by 8
  13. 2021Leveraging Time Irreversibility with Order-Contrastive Pre-trainingcited by 7
  14. 2023TemporAI: Facilitating Machine Learning Innovation in Time Domain Tasks for Medicinecited by 5
  15. 2021Explainable Health Risk Predictor with Transformer-based Medicare Claim Encodercited by 4
  16. 2021Generalized and Transferable Patient Language Representation for Phenotyping with Limited Datacited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 11; 2025: 20; 2026: 14.

46 papers.

Sort:

By area:

  1. 2024Zero Shot Health Trajectory Prediction Using Transformercited by 77
  2. 2024Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRscited by 39
  3. 2025Foundation Model of Electronic Medical Records for Adaptive Risk Estimationcited by 19
  4. 2023Event-Based Contrastive Learning for Medical Time Seriescited by 13
  5. 2024HoTPP Benchmark: Are We Good at the Long Horizon Events Forecasting?cited by 13
  6. 2025A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMscited by 13
  7. 2025Exploring Scaling Laws for EHR Foundation Modelscited by 12
  8. 2025Foundation models for electronic health records: representation dynamics and transferabilitycited by 11
  9. 2024ACES: Automatic Cohort Extraction System for Event-Stream Datasetscited by 9
  10. 2024Multimodal Banking Dataset: Understanding Client Needs through Event Sequencescited by 9
  11. 2025A Collection of Innovations in Medical AI for patient records in 2024cited by 7
  12. 2024Detecting the Future: All-at-Once Event Sequence Forecasting with Horizon Matchingcited by 6
34 more
  1. 2025FoMoH: A clinically meaningful foundation model evaluation for structured electronic health recordscited by 6
  2. 2024Generalized Prompt Tuning: Adapting Frozen Univariate Time Series Foundation Models for Multivariate Healthcare Time Seriescited by 5
  3. 2025An Investigation of Memorization Risk in Healthcare Foundation Modelscited by 5
  4. 2026One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Modelscited by 5
  5. 2024meds_reader: A fast and efficient EHR processing librarycited by 4
  6. 2025CEHR-XGPT: A Scalable Multi-Task Foundation Model for Electronic Health Recordscited by 4
  7. 2024Recent Advances, Applications, and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2023 Symposiumcited by 3
  8. 2025EHR2Path: Comprehensive Pathway-Level Modeling of Longitudinal Patient Trajectories from Multimodal Electronic Health Recordscited by 3
  9. 2025Foundation Models for Clinical Records at Health System Scalecited by 3
  10. 2025Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation modelscited by 3
  11. 2025HT-Transformer: Event Sequences Classification by Accumulating Prefix Information with History Tokenscited by 3
  12. 2025Count-Based Approaches Remain Strong: A Benchmark Against Transformer and LLM Pipelines on Structured EHRcited by 3
  13. 2026The Taxonomies, Training, and Applications of Event Stream Modelling for Electronic Health Recordscited by 3
  14. 2026Scaling Recurrence-aware Foundation Models for Clinical Records via Next-Visit Predictioncited by 3
  15. 2026Efficient Generative Prediction for EHR Foundation Models: The SCOPE and REACH Estimatorscited by 2
  16. 2026Learning Clinical Representations Under Systematic Distribution Shiftcited by 2
  17. 2026Discriminative Representation Learning for Clinical Predictioncited by 2
  18. 2024ESQA: Event Sequences Question Answeringcited by 1
  19. 2024A Simple Baseline for Predicting Events with Auto-Regressive Tabular Transformerscited by 1
  20. 2025Temporal Entailment Pretraining for Clinical Language Models over EHR Datacited by 1
  21. 2026Uncertainty-Aware Foundation Models for Clinical Datacited by 1
  22. 2026Foundation Models to Unlock Real-World Evidence from Nationwide Medical Claimscited by 1
  23. 2025Towards Unified Approaches in Self-Supervised Event Stream Modeling: Progress and Prospectscited by 0
  24. 2025Time After Time: Deep-Q Effect Estimation for Interventions on When and What to docited by 0
  25. 2025Scaling Electronic Health Record Foundation Models for Population Health Managementcited by 0
  26. 2025A Generative Approach for Semantic Auditing of Electronic Health Recordscited by 0
  27. 2025EMR-AGENT: Automating Cohort and Feature Extraction from EMR Databasescited by 0
  28. 2025Coefficient of Variation Masking: A Volatility-Aware Strategy for EHR Foundation Modelscited by 0
  29. 2026NEST: Nested Event Stream Transformer for Sequences of Multisetscited by 0
  30. 2026Representation Before Training: A Practical Benchmark for Generative Medical Event Model Tokenizationcited by 0
  31. 2026SOHET: Sequence Of Heterogeneous Events Transformer with Self-Supervised Pre-Trainingcited by 0
  32. 2026Federated generative event models for tokenized electronic health recordscited by 0
  33. 2026Physiological World Models for Human State Transitionscited by 0
  34. 2026Reinforcement Learning over Patient Trajectories for Clinical Reasoning in EHR Foundation Modelscited by 0