# MOTOR: A Time-To-Event Foundation Model For Structured Medical Records

arXiv: https://arxiv.org/abs/2301.03150 · HTML: https://mlsanity.com/paper/2301.03150/

## Key facts

- Cited by 78 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 77 references that are in the map.
- First posted on arXiv 9 Jan 2023.

## Builds on (most cited first, 28 of 28)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Efficient Content-Based Sparse Attention with Routing Transformers](https://mlsanity.com/paper/2003.05997/) (2020)
- [Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction](https://mlsanity.com/paper/2005.12833/) (2020)
- [DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Network](https://mlsanity.com/paper/1606.00931/) (2016)
- [RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism](https://mlsanity.com/paper/1608.05745/) (2016)
- [BEHRT: Transformer for Electronic Health Records](https://mlsanity.com/paper/1907.09538/) (2019)
- [Semi-supervised Sequence Learning](https://mlsanity.com/paper/1511.01432/) (2015)
- [Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Data](https://mlsanity.com/paper/2001.05295/) (2020)
- [CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks](https://mlsanity.com/paper/2111.08585/) (2021)
- [Continuous and Discrete-Time Survival Prediction with Neural Networks](https://mlsanity.com/paper/1910.06724/) (2019)
- [Deep Survival Machines: Fully Parametric Survival Regression and Representation Learning for Censored Data with Competing Risks](https://mlsanity.com/paper/2003.01176/) (2020)
- [Deep Recurrent Survival Analysis](https://mlsanity.com/paper/1809.02403/) (2018)
- [SurvTRACE: Transformers for Survival Analysis with Competing Events](https://mlsanity.com/paper/2110.00855/) (2021)
- [Robust and Efficient Medical Imaging with Self-Supervision](https://mlsanity.com/paper/2205.09723/) (2022)
- [Countdown Regression: Sharp and Calibrated Survival Predictions](https://mlsanity.com/paper/1806.08324/) (2018)
- [Improving Palliative Care with Deep Learning](https://mlsanity.com/paper/1711.06402/) (2017)
- [Gaussian Processes for Survival Analysis](https://mlsanity.com/paper/1611.00817/) (2016)
- [Semi-Structured Deep Piecewise Exponential Models](https://mlsanity.com/paper/2011.05824/) (2020)
- [Churn Prediction in Mobile Social Games: Towards a Complete Assessment Using Survival Ensembles](https://mlsanity.com/paper/1710.02264/) (2017)
- [Disease-Atlas: Navigating Disease Trajectories with Deep Learning](https://mlsanity.com/paper/1803.10254/) (2018)
- [Disparate Censorship & Undertesting: A Source of Label Bias in Clinical Machine Learning](https://mlsanity.com/paper/2208.01127/) (2022)
- [Combining parametric, semi-parametric, and non-parametric survival models with stacked survival models](https://mlsanity.com/paper/1309.7936/) (2013)

## Built on by (most cited first, 30 of 78)

- [Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRs](https://mlsanity.com/paper/2412.16178/) (2024)
- [A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Records](https://mlsanity.com/paper/2311.11483/) (2023)
- [An Introduction to Deep Survival Analysis Models for Predicting Time-to-Event Outcomes](https://mlsanity.com/paper/2410.01086/) (2024)
- [A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMs](https://mlsanity.com/paper/2504.14657/) (2025)
- [Foundation models for electronic health records: representation dynamics and transferability](https://mlsanity.com/paper/2504.10422/) (2025)
- [Tabular Foundation Models Can Do Survival Analysis](https://mlsanity.com/paper/2601.22259/) (2026)
- [Enhancing Antibiotic Stewardship using a Natural Language Approach for Better Feature Representation](https://mlsanity.com/paper/2405.20419/) (2024)
- [Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction](https://mlsanity.com/paper/2501.17326/) (2025)
- [Time-to-Event Pretraining for 3D Medical Imaging](https://mlsanity.com/paper/2411.09361/) (2024)
- [A Collection of Innovations in Medical AI for patient records in 2024](https://mlsanity.com/paper/2503.05768/) (2025)
- [Position: Stop Chasing the C-index when Evaluating Survival Analysis Models](https://mlsanity.com/paper/2506.02075/) (2025)
- [Serialized EHR make for good text representations](https://mlsanity.com/paper/2510.13843/) (2025)
- [FoMoH: A clinically meaningful foundation model evaluation for structured electronic health records](https://mlsanity.com/paper/2505.16941/) (2025)
- [Building the EHR Foundation Model via Next Event Prediction](https://mlsanity.com/paper/2509.25591/) (2025)
- [Fusion or Confusion? Multimodal Complexity Is Not All You Need](https://mlsanity.com/paper/2512.22991/) (2025)
- [One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models](https://mlsanity.com/paper/2602.00541/) (2026)
- [CEHR-XGPT: A Scalable Multi-Task Foundation Model for Electronic Health Records](https://mlsanity.com/paper/2509.03643/) (2025)
- [Semantic Self-Distillation for Language Model Uncertainty](https://mlsanity.com/paper/2602.04577/) (2026)
- [Medical world models in healthcare: foundations, applications, and challenges for trustworthy clinical translation](https://mlsanity.com/paper/2607.25242/) (2026)
- [Uncertainty Quantification for Clinical Outcome Predictions with (Large) Language Models](https://mlsanity.com/paper/2411.03497/) (2024)
- [Will Large Language Models Transform Clinical Prediction?](https://mlsanity.com/paper/2505.18246/) (2025)
- [EHR2Path: Comprehensive Pathway-Level Modeling of Longitudinal Patient Trajectories from Multimodal Electronic Health Records](https://mlsanity.com/paper/2506.04831/) (2025)
- [Time-Aware Attention for Enhanced Electronic Health Records Modeling](https://mlsanity.com/paper/2507.14847/) (2025)
- [Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation models](https://mlsanity.com/paper/2507.22798/) (2025)
- [Count-Based Approaches Remain Strong: A Benchmark Against Transformer and LLM Pipelines on Structured EHR](https://mlsanity.com/paper/2511.00782/) (2025)
- [The Taxonomies, Training, and Applications of Event Stream Modelling for Electronic Health Records](https://mlsanity.com/paper/2603.14003/) (2026)
- [SurvUnc: A Meta-Model Based Uncertainty Quantification Framework for Survival Analysis](https://mlsanity.com/paper/2505.14803/) (2025)
- [Dementia-R1: Reinforced Pretraining and Reasoning from Unstructured Clinical Notes for Real-World Dementia Prognosis](https://mlsanity.com/paper/2601.03018/) (2026)
- [Learning Clinical Representations Under Systematic Distribution Shift](https://mlsanity.com/paper/2603.07348/) (2026)
- [LLMs can construct powerful representations and streamline sample-efficient supervised learning](https://mlsanity.com/paper/2603.11679/) (2026)

## Abstract (arXiv)

We present a self-supervised, time-to-event (TTE) foundation model called MOTOR (Many Outcome Time Oriented Representations) which is pretrained on timestamped sequences of events in electronic health records (EHR) and health insurance claims. TTE models are used for estimating the probability distribution of the time until a specific event occurs, which is an important task in medical settings. TTE models provide many advantages over classification using fixed time horizons, including naturally handling censored observations, but are challenging to train with limited labeled data. MOTOR addresses this challenge by pretraining on up to 55M patient records (9B clinical events). We evaluate MOTOR's transfer learning performance on 19 tasks, across 3 patient databases (a private EHR system, MIMIC-IV, and Merative claims data). Task-specific models adapted from MOTOR improve time-dependent C statistics by 4.6% over state-of-the-art, improve label efficiency by up to 95% ,and are more robust to temporal distributional shifts. We further evaluate cross-site portability by adapting our MOTOR foundation model for six prediction tasks on the MIMIC-IV dataset, where it outperforms all baselines. MOTOR is the first foundation model for medical TTE predictions and we release a 143M parameter pretrained model for research use at [redacted URL].