# Multimodal Spatial Attention Module for Targeting Multimodal PET-CT Lung Tumor Segmentation

arXiv: https://arxiv.org/abs/2007.14728 · HTML: https://mlsanity.com/paper/2007.14728/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jul 2020.

## Built on by (most cited first, 19 of 19)

- [ScribbleVC: Scribble-supervised Medical Image Segmentation with Vision-Class Embedding](https://mlsanity.com/paper/2307.16226/) (2023)
- [Vision Transformers in Medical Computer Vision -- A Contemplative Retrospection](https://mlsanity.com/paper/2203.15269/) (2022)
- [Modality specific U-Net variants for biomedical image segmentation: A survey](https://mlsanity.com/paper/2107.04537/) (2021)
- [Mirror U-Net: Marrying Multimodal Fission with Multi-task Learning for Semantic Segmentation in Medical Imaging](https://mlsanity.com/paper/2303.07126/) (2023)
- [Multimodal Data Integration for Precision Oncology: Challenges and Future Directions](https://mlsanity.com/paper/2406.19611/) (2024)
- [Coarse-to-Fine Covid-19 Segmentation via Vision-Language Alignment](https://mlsanity.com/paper/2303.00279/) (2023)
- [H-DenseFormer: An Efficient Hybrid Densely Connected Transformer for Multimodal Tumor Segmentation](https://mlsanity.com/paper/2307.01486/) (2023)
- [From Classical Machine Learning to Emerging Foundation Models: Review on Multimodal Data Integration for Cancer Research](https://mlsanity.com/paper/2507.09028/) (2025)
- [Comprehensive framework for evaluation of deep neural networks in detection and quantification of lymphoma from PET/CT images: clinical insights, pitfalls, and observer agreement analyses](https://mlsanity.com/paper/2311.09614/) (2023)
- [ISA-Net: Improved spatial attention network for PET-CT tumor segmentation](https://mlsanity.com/paper/2211.02256/) (2022)
- [Cross-Modal Interactive Perception Network with Mamba for Lung Tumor Segmentation in PET-CT Images](https://mlsanity.com/paper/2503.17261/) (2025)
- [Automatic Tumor Segmentation via False Positive Reduction Network for Whole-Body Multi-Modal PET/CT Images](https://mlsanity.com/paper/2209.07705/) (2022)
- [Opportunities and challenges in the application of large artificial intelligence models in radiology](https://mlsanity.com/paper/2403.16112/) (2024)
- [Localization-Infused Vision-Language Semantic Fusion for Text-Guided Medical Image Segmentation](https://mlsanity.com/paper/2607.16327/) (2026)
- [Teacher-Student Architecture for Mixed Supervised Lung Tumor Segmentation](https://mlsanity.com/paper/2112.11541/) (2021)
- [Hyper-Connected Transformer Network for Multi-Modality PET-CT Segmentation](https://mlsanity.com/paper/2210.15808/) (2022)
- [3D PETCT Tumor Lesion Segmentation via GCN Refinement](https://mlsanity.com/paper/2302.12571/) (2023)
- [HyperMM : Robust Multimodal Learning with Varying-sized Inputs](https://mlsanity.com/paper/2407.20768/) (2024)
- [Pretrained, Curriculum-Tuned, and Ensembled: A Tracer-Aware Interactive Segmentation Pipeline for AutoPET V](https://mlsanity.com/paper/2608.30844/) (2026)

## Abstract (arXiv)

Multimodal positron emission tomography-computed tomography (PET-CT) is used routinely in the assessment of cancer. PET-CT combines the high sensitivity for tumor detection with PET and anatomical information from CT. Tumor segmentation is a critical element of PET-CT but at present, there is not an accurate automated segmentation method. Segmentation tends to be done manually by different imaging experts and it is labor-intensive and prone to errors and inconsistency. Previous automated segmentation methods largely focused on fusing information that is extracted separately from the PET and CT modalities, with the underlying assumption that each modality contains complementary information. However, these methods do not fully exploit the high PET tumor sensitivity that can guide the segmentation. We introduce a multimodal spatial attention module (MSAM) that automatically learns to emphasize regions (spatial areas) related to tumors and suppress normal regions with physiologic high-uptake. The resulting spatial attention maps are subsequently employed to target a convolutional neural network (CNN) for segmentation of areas with higher tumor likelihood. Our MSAM can be applied to common backbone architectures and trained end-to-end. Our experimental results on two clinical PET-CT datasets of non-small cell lung cancer (NSCLC) and soft tissue sarcoma (STS) validate the effectiveness of the MSAM in these different cancer types. We show that our MSAM, with a conventional U-Net backbone, surpasses the state-of-the-art lung tumor segmentation approach by a margin of 7.6% in Dice similarity coefficient (DSC).