# Co-Learning Feature Fusion Maps from PET-CT Images of Lung Cancer

arXiv: https://arxiv.org/abs/1810.02492 · HTML: https://mlsanity.com/paper/1810.02492/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2018.

## Built on by (most cited first, 14 of 14)

- [Multimodal Optimal Transport-based Co-Attention Transformer with Global Structure Consistency for Survival Prediction](https://mlsanity.com/paper/2306.08330/) (2023)
- [Overview of the HECKTOR Challenge at MICCAI 2021: Automatic Head and Neck Tumor Segmentation and Outcome Prediction in PET/CT Images](https://mlsanity.com/paper/2201.04138/) (2022)
- [Multimodal Data Integration for Precision Oncology: Challenges and Future Directions](https://mlsanity.com/paper/2406.19611/) (2024)
- [GTP-4o: Modality-prompted Heterogeneous Graph Learning for Omni-modal Biomedical Representation](https://mlsanity.com/paper/2407.05540/) (2024)
- [ISA-Net: Improved spatial attention network for PET-CT tumor segmentation](https://mlsanity.com/paper/2211.02256/) (2022)
- [Cross-Modal Interactive Perception Network with Mamba for Lung Tumor Segmentation in PET-CT Images](https://mlsanity.com/paper/2503.17261/) (2025)
- [AI-Based Detection, Classification and Prediction/Prognosis in Medical Imaging: Towards Radiophenomics](https://mlsanity.com/paper/2110.10332/) (2021)
- [DXM-TransFuse U-net: Dual Cross-Modal Transformer Fusion U-net for Automated Nerve Identification](https://mlsanity.com/paper/2202.13304/) (2022)
- [A cascaded deep network for automated tumor detection and segmentation in clinical PET imaging of diffuse large B-cell lymphoma](https://mlsanity.com/paper/2403.07092/) (2024)
- [Local Clustering for Lung Cancer Image Classification via Sparse Solution Technique](https://mlsanity.com/paper/2407.08800/) (2024)
- [Hyper-Connected Transformer Network for Multi-Modality PET-CT Segmentation](https://mlsanity.com/paper/2210.15808/) (2022)
- [Deep Ensembling with Multimodal Image Fusion for Efficient Classification of Lung Cancer](https://mlsanity.com/paper/2502.00078/) (2025)
- [MSAD-Net: Multiscale and Spatial Attention-based Dense Network for Lung Cancer Classification](https://mlsanity.com/paper/2504.14626/) (2025)
- [Multimodal Slice Interaction Network Enhanced by Transfer Learning for Precise Segmentation of Internal Gross Tumor Volume in Lung Cancer PET/CT Imaging](https://mlsanity.com/paper/2509.22841/) (2025)

## Abstract (arXiv)

The analysis of multi-modality positron emission tomography and computed tomography (PET-CT) images for computer aided diagnosis applications requires combining the sensitivity of PET to detect abnormal regions with anatomical localization from CT. Current methods for PET-CT image analysis either process the modalities separately or fuse information from each modality based on knowledge about the image analysis task. These methods generally do not consider the spatially varying visual characteristics that encode different information across the different modalities, which have different priorities at different locations. For example, a high abnormal PET uptake in the lungs is more meaningful for tumor detection than physiological PET uptake in the heart. Our aim is to improve fusion of the complementary information in multi-modality PET-CT with a new supervised convolutional neural network (CNN) that learns to fuse complementary information for multi-modality medical image analysis. Our CNN first encodes modality-specific features and then uses them to derive a spatially varying fusion map that quantifies the relative importance of each modality's features across different spatial locations. These fusion maps are then multiplied with the modality-specific feature maps to obtain a representation of the complementary multi-modality information at different locations, which can then be used for image analysis. We evaluated the ability of our CNN to detect and segment multiple regions with different fusion requirements using a dataset of PET-CT images of lung cancer. We compared our method to baseline techniques for multi-modality image fusion and segmentation. Our findings show that our CNN had a significantly higher foreground detection accuracy (99.29%, p < 0.05) than the fusion baselines and a significantly higher Dice score (63.85%) than recent PET-CT tumor segmentation methods.