mlsanityData to 7 Oct 2026

MOTOR: A Time-To-Event Foundation Model For Structured Medical Records

arXiv 2301.03150 ↗First posted 9 Jan 2023cs.LGcited by 75

Not on the map. Why?

Abstract (arXiv)

We present a self-supervised, time-to-event (TTE) foundation model called MOTOR (Many Outcome Time Oriented Representations) which is pretrained on timestamped sequences of events in electronic health records (EHR) and health insurance claims. TTE models are used for estimating the probability distribution of the time until a specific event occurs, which is an important task in medical settings. TTE models provide many advantages over classification using fixed time horizons, including naturally handling censored observations, but are challenging to train with limited labeled data. MOTOR addresses this challenge by pretraining on up to 55M patient records (9B clinical events). We evaluate MOTOR's transfer learning performance on 19 tasks, across 3 patient databases (a private EHR system, MIMIC-IV, and Merative claims data). Task-specific models adapted from MOTOR improve time-dependent C statistics by 4.6% over state-of-the-art, improve label efficiency by up to 95% ,and are more robust to temporal distributional shifts. We further evaluate cross-site portability by adapting our MOTOR foundation model for six prediction tasks on the MIMIC-IV dataset, where it outperforms all baselines. MOTOR is the first foundation model for medical TTE predictions and we release a 143M parameter pretrained model for research use at [redacted URL].

Key facts. Cited by 75 papers in the map (data through 7 Oct 2026). Builds on 28 of its 77 references that are in the map. First posted on arXiv 9 Jan 2023.

Builds on

28 of 77 references.

References · 28

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  5. 2018Deep contextualized word representationscited by 2,285
  6. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  7. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 548
  8. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  9. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  10. 2016DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Networkcited by 354
  11. 2016RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanismcited by 344
  12. 2019BEHRT: Transformer for Electronic Health Recordscited by 258
16 more
  1. 2015Semi-supervised Sequence Learningcited by 198
  2. 2020Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Datacited by 85
  3. 2021CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction taskscited by 75
  4. 2020Deep Survival Machines: Fully Parametric Survival Regression and Representation Learning for Censored Data with Competing Riskscited by 70
  5. 2019Continuous and Discrete-Time Survival Prediction with Neural Networkscited by 68
  6. 2018Deep Recurrent Survival Analysiscited by 61
  7. 2021SurvTRACE: Transformers for Survival Analysis with Competing Eventscited by 40
  8. 2022Robust and Efficient Medical Imaging with Self-Supervisioncited by 40
  9. 2018Countdown Regression: Sharp and Calibrated Survival Predictionscited by 30
  10. 2017Improving Palliative Care with Deep Learningcited by 28
  11. 2016Gaussian Processes for Survival Analysiscited by 14
  12. 2020Semi-Structured Deep Piecewise Exponential Modelscited by 13
  13. 2017Churn Prediction in Mobile Social Games: Towards a Complete Assessment Using Survival Ensemblescited by 11
  14. 2018Disease-Atlas: Navigating Disease Trajectories with Deep Learningcited by 8
  15. 2022Disparate Censorship & Undertesting: A Source of Label Bias in Clinical Machine Learningcited by 7
  16. 2013Combining parametric, semi-parametric, and non-parametric survival models with stacked survival modelscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 23; 2026: 46.

75 papers.

Sort:

By area:

  1. 2024Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRscited by 39
  2. 2023A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Recordscited by 30
  3. 2024An Introduction to Deep Survival Analysis Models for Predicting Time-to-Event Outcomescited by 16
  4. 2025A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMscited by 13
  5. 2025Foundation models for electronic health records: representation dynamics and transferabilitycited by 11
  6. 2024Enhancing Antibiotic Stewardship using a Natural Language Approach for Better Feature Representationcited by 8
  7. 2024Time-to-Event Pretraining for 3D Medical Imagingcited by 7
  8. 2025A Collection of Innovations in Medical AI for patient records in 2024cited by 7
  9. 2025Serialized EHR make for good text representationscited by 7
  10. 2025Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Predictioncited by 6
  11. 2025FoMoH: A clinically meaningful foundation model evaluation for structured electronic health recordscited by 6
  12. 2025Position: Stop Chasing the C-index when Evaluating Survival Analysis Modelscited by 5
38 more
  1. 2025Building the EHR Foundation Model via Next Event Predictioncited by 5
  2. 2026Tabular Foundation Models Can Do Survival Analysiscited by 5
  3. 2026One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Modelscited by 5
  4. 2025CEHR-XGPT: A Scalable Multi-Task Foundation Model for Electronic Health Recordscited by 4
  5. 2025Fusion or Confusion? Multimodal Complexity Is Not All You Needcited by 4
  6. 2026Semantic Self-Distillation for Language Model Uncertaintycited by 4
  7. 2024Uncertainty Quantification for Clinical Outcome Predictions with (Large) Language Modelscited by 3
  8. 2025Will Large Language Models Transform Clinical Prediction?cited by 3
  9. 2025Time-Aware Attention for Enhanced Electronic Health Records Modelingcited by 3
  10. 2025Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation modelscited by 3
  11. 2026The Taxonomies, Training, and Applications of Event Stream Modelling for Electronic Health Recordscited by 3
  12. 2026Medical world models in healthcare: foundations, applications, and challenges for trustworthy clinical translationcited by 3
  13. 2025SurvUnc: A Meta-Model Based Uncertainty Quantification Framework for Survival Analysiscited by 2
  14. 2025EHR2Path: Comprehensive Pathway-Level Modeling of Longitudinal Patient Trajectories from Multimodal Electronic Health Recordscited by 2
  15. 2025Count-Based Approaches Remain Strong: A Benchmark Against Transformer and LLM Pipelines on Structured EHRcited by 2
  16. 2026Dementia-R1: Reinforced Pretraining and Reasoning from Unstructured Clinical Notes for Real-World Dementia Prognosiscited by 2
  17. 2026Learning Clinical Representations Under Systematic Distribution Shiftcited by 2
  18. 2026LLMs can construct powerful representations and streamline sample-efficient supervised learningcited by 2
  19. 2026Discriminative Representation Learning for Clinical Predictioncited by 2
  20. 2026PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Modelscited by 2
  21. 2025Temporal Entailment Pretraining for Clinical Language Models over EHR Datacited by 1
  22. 2025Federated Timeline Synthesis: Scalable and Private Methodology For Model Training and Deploymentcited by 1
  23. 2026Personalization of Large Foundation Models for Health Interventionscited by 1
  24. 2026Learning Longitudinal Health Representations from EHR and Wearable Datacited by 1
  25. 2026TwinWeaver: An LLM-Based Foundation Model Framework for Pan-Cancer Digital Twinscited by 1
  26. 2026Foundation Models to Unlock Real-World Evidence from Nationwide Medical Claimscited by 1
  27. 2025Towards Unified Approaches in Self-Supervised Event Stream Modeling: Progress and Prospectscited by 0
  28. 2025Deep Causal Behavioral Policy Learning: Applications to Healthcarecited by 0
  29. 2025Scaling Electronic Health Record Foundation Models for Population Health Managementcited by 0
  30. 2025Learning Survival Models with Right-Censored Reporting Delayscited by 0
  31. 2025Toward Scalable Early Cancer Detection: Evaluating EHR-Based Predictive Models Against Traditional Screening Criteriacited by 0
  32. 2026NEST: Nested Event Stream Transformer for Sequences of Multisetscited by 0
  33. 2026Dense Feature Learning via Linear Structure Preservation in Medical Datacited by 0
  34. 2026Risk Horizons: Structured Hypothesis Spaces for Longitudinal Clinical Predictioncited by 0
  35. 2026EveryQuery: A Promptable Foundation Model for Clinical Prediction Tasks over Electronic Health Recordscited by 0
  36. 2026Representation Before Training: A Practical Benchmark for Generative Medical Event Model Tokenizationcited by 0
  37. 2026FlatASCEND: Autoregressive Clinical Sequence Generation with Continuous Time Prediction and Association-Based Pharmacological Testingcited by 0
  38. 2026Sparse Autoencoder Decomposition of Clinical Sequence Model Representations: Feature Complexity, Task Specialisation, and Mortality Predictioncited by 0

50 most cited of 75 All 75 (JSON)