# Merging-Diverging Hybrid Transformer Networks for Survival Prediction in Head and Neck Cancer

arXiv: https://arxiv.org/abs/2307.03427 · HTML: https://mlsanity.com/paper/2307.03427/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 7 Jul 2023.

## Builds on (most cited first, 16 of 16)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images](https://mlsanity.com/paper/1808.08114/) (2018)
- [A Scalable Discrete-Time Survival Model for Neural Networks](https://mlsanity.com/paper/1805.00917/) (2018)
- [Overview of the HECKTOR Challenge at MICCAI 2021: Automatic Head and Neck Tumor Segmentation and Outcome Prediction in PET/CT Images](https://mlsanity.com/paper/2201.04138/) (2022)
- [Biomedical image analysis competitions: The state of current participation practice](https://mlsanity.com/paper/2212.08568/) (2022)
- [From Hand-Crafted to Deep Learning-based Cancer Radiomics: Challenges and Opportunities](https://mlsanity.com/paper/1808.07954/) (2018)
- [TMSS: An End-to-End Transformer-based Multimodal Network for Segmentation and Survival Prediction](https://mlsanity.com/paper/2209.05036/) (2022)
- [PHTrans: Parallelly Aggregating Global and Local Representations for Medical Image Segmentation](https://mlsanity.com/paper/2203.04568/) (2022)
- [An Ensemble Approach for Patient Prognosis of Head and Neck Tumor Using Multimodal Data](https://mlsanity.com/paper/2202.12537/) (2022)
- [DeepMTS: Deep Multi-task Learning for Survival Prediction in Patients with Advanced Nasopharyngeal Carcinoma using Pretreatment PET/CT](https://mlsanity.com/paper/2109.07711/) (2021)
- [Prediction of 5-year Progression-Free Survival in Advanced Nasopharyngeal Carcinoma with Pretreatment PET/CT using Multi-Modality Deep Learning-based Radiomics](https://mlsanity.com/paper/2103.05220/) (2021)
- [Fusing Modalities by Multiplexed Graph Neural Networks for Outcome Prediction in Tuberculosis](https://mlsanity.com/paper/2210.14377/) (2022)
- [Radiomics-enhanced Deep Multi-task Learning for Outcome Prediction in Head and Neck Cancer](https://mlsanity.com/paper/2211.05409/) (2022)
- [MMMNA-Net for Overall Survival Time Prediction of Brain Tumor Patients](https://mlsanity.com/paper/2206.06267/) (2022)

## Built on by (most cited first, 9 of 9)

- [SurvRNC: Learning Ordered Representations for Survival Prediction using Rank-N-Contrast](https://mlsanity.com/paper/2403.10603/) (2024)
- [Multimodal Data Integration for Precision Oncology: Challenges and Future Directions](https://mlsanity.com/paper/2406.19611/) (2024)
- [Full-resolution MLPs Empower Medical Dense Prediction](https://mlsanity.com/paper/2311.16707/) (2023)
- [Survival Prediction in Lung Cancer through Multi-Modal Representation Learning](https://mlsanity.com/paper/2409.20179/) (2024)
- [RobSurv: Vector Quantization-Based Multi-Modal Learning for Robust Cancer Survival Prediction](https://mlsanity.com/paper/2505.02529/) (2025)
- [Toward Next-generation Medical Vision Backbones: Modeling Finer-grained Long-range Visual Dependency](https://mlsanity.com/paper/2509.11328/) (2025)
- [Discovering Subgroups with Exceptional Survival Characteristics](https://mlsanity.com/paper/2602.22179/) (2026)
- [ChronoSurv: A Clinical Pathway-Guided Graph Framework for Multimodal Survival Analysis](https://mlsanity.com/paper/2606.19140/) (2026)
- [CoMLP: Cooperatively-Gated MLPs for Fine-Grained Cross-Modal Information Fusion in Medical Image Segmentation](https://mlsanity.com/paper/2609.04781/) (2026)

## Abstract (arXiv)

Survival prediction is crucial for cancer patients as it provides early prognostic information for treatment planning. Recently, deep survival models based on deep learning and medical images have shown promising performance for survival prediction. However, existing deep survival models are not well developed in utilizing multi-modality images (e.g., PET-CT) and in extracting region-specific information (e.g., the prognostic information in Primary Tumor (PT) and Metastatic Lymph Node (MLN) regions). In view of this, we propose a merging-diverging learning framework for survival prediction from multi-modality images. This framework has a merging encoder to fuse multi-modality information and a diverging decoder to extract region-specific information. In the merging encoder, we propose a Hybrid Parallel Cross-Attention (HPCA) block to effectively fuse multi-modality features via parallel convolutional layers and cross-attention transformers. In the diverging decoder, we propose a Region-specific Attention Gate (RAG) block to screen out the features related to lesion regions. Our framework is demonstrated on survival prediction from PET-CT images in Head and Neck (H&N) cancer, by designing an X-shape merging-diverging hybrid transformer network (named XSurv). Our XSurv combines the complementary information in PET and CT images and extracts the region-specific prognostic information in PT and MLN regions. Extensive experiments on the public dataset of HEad and neCK TumOR segmentation and outcome prediction challenge (HECKTOR 2022) demonstrate that our XSurv outperforms state-of-the-art survival prediction methods.