# An Ensemble Approach for Patient Prognosis of Head and Neck Tumor Using Multimodal Data

arXiv: https://arxiv.org/abs/2202.12537 · HTML: https://mlsanity.com/paper/2202.12537/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 17 references that are in the map.
- First posted on arXiv 25 Feb 2022.

## Builds on (most cited first, 5 of 5)

- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Deep Neural Networks for Survival Analysis Based on a Multi-Task Framework](https://mlsanity.com/paper/1801.05512/) (2018)
- [Overview of the HECKTOR Challenge at MICCAI 2021: Automatic Head and Neck Tumor Segmentation and Outcome Prediction in PET/CT Images](https://mlsanity.com/paper/2201.04138/) (2022)
- [Deep-CR MTLR: a Multi-Modal Approach for Cancer Survival Prediction with Competing Risks](https://mlsanity.com/paper/2012.05765/) (2020)
- [A Machine Learning Challenge for Prognostic Modelling in Head and Neck Cancer Using Multi-modal Data](https://mlsanity.com/paper/2101.11935/) (2021)

## Built on by (most cited first, 13 of 13)

- [TMSS: An End-to-End Transformer-based Multimodal Network for Segmentation and Survival Prediction](https://mlsanity.com/paper/2209.05036/) (2022)
- [SurvRNC: Learning Ordered Representations for Survival Prediction using Rank-N-Contrast](https://mlsanity.com/paper/2403.10603/) (2024)
- [Merging-Diverging Hybrid Transformer Networks for Survival Prediction in Head and Neck Cancer](https://mlsanity.com/paper/2307.03427/) (2023)
- [Radiomics-enhanced Deep Multi-task Learning for Outcome Prediction in Head and Neck Cancer](https://mlsanity.com/paper/2211.05409/) (2022)
- [A Multimodal and Multi-centric Head and Neck Cancer Dataset for Segmentation, Diagnosis and Outcome Prediction](https://mlsanity.com/paper/2509.00367/) (2025)
- [AdaMSS: Adaptive Multi-Modality Segmentation-to-Survival Learning for Survival Outcome Prediction from PET/CT Images](https://mlsanity.com/paper/2305.09946/) (2023)
- [Survival Prediction in Lung Cancer through Multi-Modal Representation Learning](https://mlsanity.com/paper/2409.20179/) (2024)
- [HuLP: Human-in-the-Loop for Prognosis](https://mlsanity.com/paper/2403.13078/) (2024)
- [Segmentation-Free Outcome Prediction from Head and Neck Cancer PET/CT Images: Deep Learning-Based Feature Extraction from Multi-Angle Maximum Intensity Projections (MA-MIPs)](https://mlsanity.com/paper/2405.01756/) (2024)
- [Towards Tumour Graph Learning for Survival Prediction in Head & Neck Cancer Patients](https://mlsanity.com/paper/2304.08106/) (2023)
- [Diagnosis and Prognosis of Head and Neck Cancer Patients using Artificial Intelligence](https://mlsanity.com/paper/2306.00034/) (2023)
- [Efficient Parameter Adaptation for Multi-Modal Medical Image Segmentation and Prognosis](https://mlsanity.com/paper/2504.13645/) (2025)
- [UNICON: UNIfied CONtinual Learning for Medical Foundational Models](https://mlsanity.com/paper/2508.14024/) (2025)

## Abstract (arXiv)

Accurate prognosis of a tumor can help doctors provide a proper course of treatment and, therefore, save the lives of many. Traditional machine learning algorithms have been eminently useful in crafting prognostic models in the last few decades. Recently, deep learning algorithms have shown significant improvement when developing diagnosis and prognosis solutions to different healthcare problems. However, most of these solutions rely solely on either imaging or clinical data. Utilizing patient tabular data such as demographics and patient medical history alongside imaging data in a multimodal approach to solve a prognosis task has started to gain more interest recently and has the potential to create more accurate solutions. The main issue when using clinical and imaging data to train a deep learning model is to decide on how to combine the information from these sources. We propose a multimodal network that ensembles deep multi-task logistic regression (MTLR), Cox proportional hazard (CoxPH) and CNN models to predict prognostic outcomes for patients with head and neck tumors using patients' clinical and imaging (CT and PET) data. Features from CT and PET scans are fused and then combined with patients' electronic health records for the prediction. The proposed model is trained and tested on 224 and 101 patient records respectively. Experimental results show that our proposed ensemble solution achieves a C-index of 0.72 on The HECKTOR test set that saved us the first place in prognosis task of the HECKTOR challenge. The full implementation based on PyTorch is available on urlhttps://github.com/numanai/BioMedIA-Hecktor2021.