# A Large-Scale Neutral Comparison Study of Survival Models on Low-Dimensional Data

arXiv: https://arxiv.org/abs/2406.04098 · HTML: https://mlsanity.com/paper/2406.04098/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 126 references that are in the map.
- First posted on arXiv 6 Jun 2024.

## Builds on (most cited first, 17 of 17)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Network](https://mlsanity.com/paper/1606.00931/) (2016)
- [ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R](https://mlsanity.com/paper/1508.04409/) (2015)
- [Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges](https://mlsanity.com/paper/2107.05847/) (2021)
- [Machine Learning for Survival Analysis: A Survey](https://mlsanity.com/paper/1708.04649/) (2017)
- [Deep Learning for Survival Analysis: A Review](https://mlsanity.com/paper/2305.14961/) (2023)
- [Effective Ways to Build and Evaluate Individual Survival Distributions](https://mlsanity.com/paper/1811.11347/) (2018)
- [Survival Regression with Proper Scoring Rules and Monotonic Neural Networks](https://mlsanity.com/paper/2103.14755/) (2021)
- [Survival regression with accelerated failure time model in XGBoost](https://mlsanity.com/paper/2006.04920/) (2020)
- [Countdown Regression: Sharp and Calibrated Survival Predictions](https://mlsanity.com/paper/1806.08324/) (2018)
- [Smoothing parameter and model selection for general smooth models](https://mlsanity.com/paper/1511.03864/) (2015)
- [SurvSet: An open-source time-to-event dataset repository](https://mlsanity.com/paper/2203.03094/) (2022)
- [Large-scale benchmark study of survival prediction methods using multi-omics data](https://mlsanity.com/paper/2003.03621/) (2020)
- [Avoiding C-hacking when evaluating survival distribution predictions with discrimination measures](https://mlsanity.com/paper/2112.04828/) (2021)
- [Accelerated and interpretable oblique random survival forests](https://mlsanity.com/paper/2208.01129/) (2022)
- [When Are Scoring Rules Proper? Bridging Theory and Practice in Survival Model Evaluation](https://mlsanity.com/paper/2212.05260/) (2022)

## Built on by (most cited first, 5 of 5)

- [Comprehensive Benchmarking of Machine Learning Methods for Risk Prediction Modelling from Large-Scale Survival Data: A UK Biobank Study](https://mlsanity.com/paper/2503.08870/) (2025)
- [Reduction Techniques for Survival Analysis](https://mlsanity.com/paper/2508.05715/) (2025)
- [GRAFT: Decoupling Ranking and Calibration for Survival Analysis](https://mlsanity.com/paper/2602.07884/) (2026)
- [A Unified Survival Benchmark for Temporal Dropout Risk Prediction in Learning Analytics](https://mlsanity.com/paper/2604.08870/) (2026)
- [A reproducible and extensible framework for benchmarking competing risks survival models](https://mlsanity.com/paper/2608.00271/) (2026)

## Abstract (arXiv)

This work presents the first large-scale neutral benchmark experiment focused on single-event, right-censored, low-dimensional survival data. Benchmark experiments are essential in methodological research to scientifically compare new and existing model classes through proper empirical evaluation. Existing benchmarks in the survival literature are smaller in scale regarding the number of used datasets and extent of empirical evaluation. They often lack appropriate tuning or evaluation procedures, while other comparison studies focus on qualitative reviews rather than quantitative comparisons. This comprehensive study aims to fill the gap by neutrally evaluating a broad range of methods and providing generalizable guidelines for practitioners. We benchmark 19 models, ranging from classical statistical approaches to many common machine learning methods, on 34 publicly available datasets. The benchmark tunes models using both a discrimination measure (Harrell's C-index) and a scoring rule (Integrated Survival Brier Score), and evaluates them across six metrics covering discrimination, calibration, and overall predictive performance. Despite superior average ranks in overall predictive performance from individual learners like oblique random survival forests and likelihood-based boosting, and better discrimination rankings from multiple boosting- and tree-based methods as well as parametric survival models, no method significantly outperforms the commonly used Cox proportional hazards model for either tuning measure. We conclude that for predictive purposes in the standard survival analysis setting of low-dimensional, right-censored data, the Cox Proportional Hazards model remains a simple and robust method, sufficient for most practitioners. All code, data, and results are publicly available on GitHub https://github.com/slds-lmu/paper_2023_survival_benchmark