# Multi-stage intermediate fusion for multimodal learning to classify non-small cell lung cancer subtypes from CT and PET

arXiv: https://arxiv.org/abs/2501.12425 · HTML: https://mlsanity.com/paper/2501.12425/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 41 references that are in the map.
- First posted on arXiv 21 Jan 2025.

## Builds on (most cited first, 5 of 5)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [A Systematic Review of Intermediate Fusion in Multimodal Deep Learning for Biomedical Applications](https://mlsanity.com/paper/2408.02686/) (2024)
- [Multimodal Explainability via Latent Shift applied to COVID-19 stratification](https://mlsanity.com/paper/2212.14084/) (2022)
- [A Deep Learning Approach for Overall Survival Prediction in Lung Cancer with Missing Values](https://mlsanity.com/paper/2307.11465/) (2023)

## Built on by (most cited first, 1 of 1)

- [Virtual Scanning for NSCLC Histology: Investigating the Discriminatory Power of Synthetic PET](https://mlsanity.com/paper/2605.02746/) (2026)

## Abstract (arXiv)

Accurate classification of histological subtypes of non-small cell lung cancer (NSCLC) is essential in the era of precision medicine, yet current invasive techniques are not always feasible and may lead to clinical complications. This study presents a multi-stage intermediate fusion approach to classify NSCLC subtypes from CT and PET images. Our method integrates the two modalities at different stages of feature extraction, using voxel-wise fusion to exploit complementary information across varying abstraction levels while preserving spatial correlations. We compare our method against unimodal approaches using only CT or PET images to demonstrate the benefits of modality fusion, and further benchmark it against early and late fusion techniques to highlight the advantages of intermediate fusion during feature extraction. Additionally, we compare our model with the only existing intermediate fusion method for histological subtype classification using PET/CT images. Our results demonstrate that the proposed method outperforms all alternatives across key metrics, with an accuracy and AUC equal to 0.724 and 0.681, respectively. This non-invasive approach has the potential to significantly improve diagnostic accuracy, facilitate more informed treatment decisions, and advance personalized care in lung cancer management.