# Zero Shot Health Trajectory Prediction Using Transformer

arXiv: https://arxiv.org/abs/2407.21124 · HTML: https://mlsanity.com/paper/2407.21124/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 38 references that are in the map.
- First posted on arXiv 30 Jul 2024.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Data](https://mlsanity.com/paper/2001.05295/) (2020)
- [Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Events](https://mlsanity.com/paper/2306.11547/) (2023)
- [DRG-LLaMA : Tuning LLaMA Model to Predict Diagnosis-related Group for Hospitalized Patients](https://mlsanity.com/paper/2309.12625/) (2023)
- [Using Sequences of Life-events to Predict Human Lives](https://mlsanity.com/paper/2306.03009/) (2023)
- [Multimodal spatiotemporal graph neural networks for improved prediction of 30-day all-cause hospital readmission](https://mlsanity.com/paper/2204.06766/) (2022)

## Built on by (most cited first, 30 of 77)

- [Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRs](https://mlsanity.com/paper/2412.16178/) (2024)
- [Foundation Model of Electronic Medical Records for Adaptive Risk Estimation](https://mlsanity.com/paper/2502.06124/) (2025)
- [EHR-R1: A Reasoning-Enhanced Foundational Language Model for Electronic Health Record Analysis](https://mlsanity.com/paper/2510.25628/) (2025)
- [Multimodal Medical Code Tokenizer](https://mlsanity.com/paper/2502.04397/) (2025)
- [Exploring Scaling Laws for EHR Foundation Models](https://mlsanity.com/paper/2505.22964/) (2025)
- [Foundation models for electronic health records: representation dynamics and transferability](https://mlsanity.com/paper/2504.10422/) (2025)
- [FoMoH: A clinically meaningful foundation model evaluation for structured electronic health records](https://mlsanity.com/paper/2505.16941/) (2025)
- [Tokenization Tradeoffs in Structured EHR Foundation Models](https://mlsanity.com/paper/2603.15644/) (2026)
- [LabTOP: A Unified Model for Lab Test Outcome Prediction on Electronic Health Records](https://mlsanity.com/paper/2502.14259/) (2025)
- [An Investigation of Memorization Risk in Healthcare Foundation Models](https://mlsanity.com/paper/2510.12950/) (2025)
- [One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models](https://mlsanity.com/paper/2602.00541/) (2026)
- [Generating Clinically Realistic EHR Data via a Hierarchy- and Semantics-Guided Transformer](https://mlsanity.com/paper/2502.20719/) (2025)
- [MedRep: Medical Concept Representation for General Electronic Health Record Foundation Models](https://mlsanity.com/paper/2504.08329/) (2025)
- [CEHR-XGPT: A Scalable Multi-Task Foundation Model for Electronic Health Records](https://mlsanity.com/paper/2509.03643/) (2025)
- [Cross-Representation Benchmarking in Time-Series Electronic Health Records for Clinical Outcome Prediction](https://mlsanity.com/paper/2510.09159/) (2025)
- [Semantic Self-Distillation for Language Model Uncertainty](https://mlsanity.com/paper/2602.04577/) (2026)
- [Retrieval-aligned Tabular Foundation Models Enable Robust Clinical Risk Prediction in Electronic Health Records Under Real-world Constraints](https://mlsanity.com/paper/2604.01841/) (2026)
- [Interpretable Next-token Prediction via the Generalized Induction Head](https://mlsanity.com/paper/2411.00066/) (2024)
- [EHR2Path: Comprehensive Pathway-Level Modeling of Longitudinal Patient Trajectories from Multimodal Electronic Health Records](https://mlsanity.com/paper/2506.04831/) (2025)
- [Foundation Models for Clinical Records at Health System Scale](https://mlsanity.com/paper/2507.00574/) (2025)
- [Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation models](https://mlsanity.com/paper/2507.22798/) (2025)
- [The Taxonomies, Training, and Applications of Event Stream Modelling for Electronic Health Records](https://mlsanity.com/paper/2603.14003/) (2026)
- [Scaling Recurrence-aware Foundation Models for Clinical Records via Next-Visit Prediction](https://mlsanity.com/paper/2603.24562/) (2026)
- [Representation Learning of Lab Values via Masked AutoEncoders](https://mlsanity.com/paper/2501.02648/) (2025)
- [One Patient, Many Contexts: Scaling Medical AI with Contextual Intelligence](https://mlsanity.com/paper/2506.10157/) (2025)
- [Efficient Generative Prediction for EHR Foundation Models: The SCOPE and REACH Estimators](https://mlsanity.com/paper/2602.03730/) (2026)
- [LLMs can construct powerful representations and streamline sample-efficient supervised learning](https://mlsanity.com/paper/2603.11679/) (2026)
- [From Token to Token Pair: Efficient Prompt Compression for Large Language Models in Clinical Prediction](https://mlsanity.com/paper/2605.11774/) (2026)
- [PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Models](https://mlsanity.com/paper/2606.24102/) (2026)
- [EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twins](https://mlsanity.com/paper/2607.08793/) (2026)

## Abstract (arXiv)

Integrating modern machine learning and clinical decision-making has great promise for mitigating healthcare's increasing cost and complexity. We introduce the Enhanced Transformer for Health Outcome Simulation (ETHOS), a novel application of the transformer deep-learning architecture for analyzing high-dimensional, heterogeneous, and episodic health data. ETHOS is trained using Patient Health Timelines (PHTs)-detailed, tokenized records of health events-to predict future health trajectories, leveraging a zero-shot learning approach. ETHOS represents a significant advancement in foundation model development for healthcare analytics, eliminating the need for labeled data and model fine-tuning. Its ability to simulate various treatment pathways and consider patient-specific factors positions ETHOS as a tool for care optimization and addressing biases in healthcare delivery. Future developments will expand ETHOS' capabilities to incorporate a wider range of data types and data sources. Our work demonstrates a pathway toward accelerated AI development and deployment in healthcare.