# SurvBETA: Ensemble-Based Survival Models Using Beran Estimators and Several Attention Mechanisms

arXiv: https://arxiv.org/abs/2412.07638 · HTML: https://mlsanity.com/paper/2412.07638/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 75 references that are in the map.
- First posted on arXiv 10 Dec 2024.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Machine Learning for Survival Analysis: A Survey](https://mlsanity.com/paper/1708.04649/) (2017)
- [Deep Learning for Survival Analysis: A Review](https://mlsanity.com/paper/2305.14961/) (2023)
- [Deep Recurrent Survival Analysis](https://mlsanity.com/paper/1809.02403/) (2018)
- [SurvTRACE: Transformers for Survival Analysis with Competing Events](https://mlsanity.com/paper/2110.00855/) (2021)
- [Survival regression with accelerated failure time model in XGBoost](https://mlsanity.com/paper/2006.04920/) (2020)
- [Unbiased split variable selection for random survival forests using maximally selected rank statistics](https://mlsanity.com/paper/1605.03391/) (2016)
- [Deep Learning for Patient-Specific Kidney Graft Survival Analysis](https://mlsanity.com/paper/1705.10245/) (2017)
- [High-Dimensional Survival Analysis: Methods and Applications](https://mlsanity.com/paper/2205.02948/) (2022)
- [On the use of Harrell's C for clinical risk prediction via random survival forests](https://mlsanity.com/paper/1507.03092/) (2015)
- [A Deep Active Survival Analysis Approach for Precision Treatment Recommendations: Application of Prostate Cancer](https://mlsanity.com/paper/1804.03280/) (2018)
- [Attention-based Random Forest and Contamination Model](https://mlsanity.com/paper/2201.02880/) (2022)
- [A weighted random survival forest](https://mlsanity.com/paper/1901.00213/) (2019)
- [Ensemble methods for survival function estimation with time-varying covariates](https://mlsanity.com/paper/2006.00567/) (2020)

## Built on by (most cited first, 1 of 1)

- [Ensemble-Based Survival Models with the Self-Attended Beran Estimator Predictions](https://mlsanity.com/paper/2506.07933/) (2025)

## Abstract (arXiv)

Many ensemble-based models have been proposed to solve machine learning problems in the survival analysis framework, including random survival forests, the gradient boosting machine with weak survival models, ensembles of the Cox models. To extend the set of models, a new ensemble-based model called SurvBETA (the Survival Beran estimator Ensemble using Three Attention mechanisms) is proposed where the Beran estimator is used as a weak learner in the ensemble. The Beran estimator can be regarded as a kernel regression model taking into account the relationship between instances. Outputs of weak learners in the form of conditional survival functions are aggregated with attention weights taking into account the distance between the analyzed instance and prototypes of all bootstrap samples. The attention mechanism is used three times: for implementation of the Beran estimators, for determining specific prototypes of bootstrap samples and for aggregating the weak model predictions. The proposed model is presented in two forms: in a general form requiring to solve a complex optimization problem for its training; in a simplified form by considering a special representation of the attention weights by means of the imprecise Huber's contamination model which leads to solving a simple optimization problem. Numerical experiments illustrate properties of the model on synthetic data and compare the model with other survival models on real data. A code implementing the proposed model is publicly available.