# SurvTRACE: Transformers for Survival Analysis with Competing Events

arXiv: https://arxiv.org/abs/2110.00855 · HTML: https://mlsanity.com/paper/2110.00855/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 51 references that are in the map.
- First posted on arXiv 2 Oct 2021.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Network](https://mlsanity.com/paper/1606.00931/) (2016)
- [Recommendations as Treatments: Debiasing Learning and Evaluation](https://mlsanity.com/paper/1602.05352/) (2016)
- [Machine Learning for Survival Analysis: A Survey](https://mlsanity.com/paper/1708.04649/) (2017)
- [Continuous and Discrete-Time Survival Prediction with Neural Networks](https://mlsanity.com/paper/1910.06724/) (2019)
- [Deep Survival Machines: Fully Parametric Survival Regression and Representation Learning for Censored Data with Competing Risks](https://mlsanity.com/paper/2003.01176/) (2020)
- [Deep Survival Analysis](https://mlsanity.com/paper/1608.02158/) (2016)
- [Deep Recurrent Survival Analysis](https://mlsanity.com/paper/1809.02403/) (2018)
- [Adversarial Time-to-Event Modeling](https://mlsanity.com/paper/1804.03184/) (2018)
- [Information Theoretic Counterfactual Learning from Missing-Not-At-Random Feedback](https://mlsanity.com/paper/2009.02623/) (2020)
- [Deep Learning for Patient-Specific Kidney Graft Survival Analysis](https://mlsanity.com/paper/1705.10245/) (2017)
- [BERTSurv: BERT-Based Survival Models for Predicting Outcomes of Trauma Patients](https://mlsanity.com/paper/2103.10928/) (2021)
- [BoXHED: Boosted eXact Hazard Estimator with Dynamic covariates](https://mlsanity.com/paper/2006.14218/) (2020)

## Built on by (most cited first, 30 of 41)

- [TransTab: Learning Transferable Tabular Transformers Across Tables](https://mlsanity.com/paper/2205.09328/) (2022)
- [MOTOR: A Time-To-Event Foundation Model For Structured Medical Records](https://mlsanity.com/paper/2301.03150/) (2023)
- [Deep Learning for Survival Analysis: A Review](https://mlsanity.com/paper/2305.14961/) (2023)
- [MediTab: Scaling Medical Tabular Data Predictors via Data Consolidation, Enrichment, and Refinement](https://mlsanity.com/paper/2305.12081/) (2023)
- [PromptEHR: Conditional Electronic Healthcare Records Generation with Prompt Learning](https://mlsanity.com/paper/2211.01761/) (2022)
- [Copula-Based Deep Survival Models for Dependent Censoring](https://mlsanity.com/paper/2306.11912/) (2023)
- [HACSurv: A Hierarchical Copula-Based Approach for Survival Analysis with Dependent Competing Risks](https://mlsanity.com/paper/2410.15180/) (2024)
- [Artificial Intelligence for In Silico Clinical Trials: A Review](https://mlsanity.com/paper/2209.09023/) (2022)
- [Survival Models: Proper Scoring Rule and Stochastic Optimization with Competing Risks](https://mlsanity.com/paper/2410.16765/) (2024)
- [Survival Seq2Seq: A Survival Model based on Sequence to Sequence Architecture](https://mlsanity.com/paper/2204.04542/) (2022)
- [TemporAI: Facilitating Machine Learning Innovation in Time Domain Tasks for Medicine](https://mlsanity.com/paper/2301.12260/) (2023)
- [Understanding the Impact of Competing Events on Heterogeneous Treatment Effect Estimation from Time-to-Event Data](https://mlsanity.com/paper/2302.12718/) (2023)
- [Contrastive Learning of Temporal Distinctiveness for Survival Analysis in Electronic Health Records](https://mlsanity.com/paper/2308.13104/) (2023)
- [Interpretable Machine Learning for Survival Analysis](https://mlsanity.com/paper/2403.10250/) (2024)
- [Toward a Well-Calibrated Discrimination via Survival Outcome-Aware Contrastive Learning](https://mlsanity.com/paper/2410.11340/) (2024)
- [Adaptive Transformer Modelling of Density Function for Nonparametric Survival Analysis](https://mlsanity.com/paper/2409.06209/) (2024)
- [MENSA: A Multi-Event Network for Survival Analysis with Trajectory-based Likelihood Estimation](https://mlsanity.com/paper/2409.06525/) (2024)
- [Beyond Cox Models: Assessing the Performance of Machine-Learning Methods in Non-Proportional Hazards and Non-Linear Survival Analysis](https://mlsanity.com/paper/2504.17568/) (2025)
- [SurvUnc: A Meta-Model Based Uncertainty Quantification Framework for Survival Analysis](https://mlsanity.com/paper/2505.14803/) (2025)
- [Survival Analysis as Imprecise Classification with Trainable Kernels](https://mlsanity.com/paper/2506.10140/) (2025)
- [Dynamic Survival Transformers for Causal Inference with Electronic Health Records](https://mlsanity.com/paper/2210.15417/) (2022)
- [Maximum Likelihood Estimation of Flexible Survival Densities with Importance Sampling](https://mlsanity.com/paper/2311.01660/) (2023)
- [SeqRisk: Transformer-augmented latent variable model for robust survival prediction with longitudinal data](https://mlsanity.com/paper/2409.12709/) (2024)
- [SurvBETA: Ensemble-Based Survival Models Using Beran Estimators and Several Attention Mechanisms](https://mlsanity.com/paper/2412.07638/) (2024)
- [IPGPhormer: Interpretable Pathology Graph-Transformer for Survival Analysis](https://mlsanity.com/paper/2508.12381/) (2025)
- [EDEN : An Event DEtection Network for the annotation of Breast Cancer recurrences in administrative claims data](https://mlsanity.com/paper/2211.08077/) (2022)
- [SurvTimeSurvival: Survival Analysis On The Patient With Multiple Visits/Records](https://mlsanity.com/paper/2311.09854/) (2023)
- [ICTSurF: Implicit Continuous-Time Survival Functions with Neural Networks](https://mlsanity.com/paper/2312.05818/) (2023)
- [Teaching Models To Survive: Proper Scoring Rule and Stochastic Optimization with Competing Risks](https://mlsanity.com/paper/2406.14085/) (2024)
- [Predicting Deterioration in Mild Cognitive Impairment with Survival Transformers, Extreme Gradient Boosting and Cox Proportional Hazard Modelling](https://mlsanity.com/paper/2409.16231/) (2024)

## Abstract (arXiv)

In medicine, survival analysis studies the time duration to events of interest such as mortality. One major challenge is how to deal with multiple competing events (e.g., multiple disease diagnoses). In this work, we propose a transformer-based model that does not make the assumption for the underlying survival distribution and is capable of handling competing events, namely SurvTRACE. We account for the implicit confounders in the observational setting in multi-events scenarios, which causes selection bias as the predicted survival probability is influenced by irrelevant factors. To sufficiently utilize the survival data to train transformers from scratch, multiple auxiliary tasks are designed for multi-task learning. The model hence learns a strong shared representation from all these tasks and in turn serves for better survival analysis. We further demonstrate how to inspect the covariate relevance and importance through interpretable attention mechanisms of SurvTRACE, which suffices to great potential in enhancing clinical trial design and new treatment development. Experiments on METABRIC, SUPPORT, and SEER data with 470k patients validate the all-around superiority of our method.