# A Tri-attention Fusion Guided Multi-modal Segmentation Network

arXiv: https://arxiv.org/abs/2111.01623 · HTML: https://mlsanity.com/paper/2111.01623/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 2 Nov 2021.

## Builds on (most cited first, 12 of 12)

- [3D MRI brain tumor segmentation using autoencoder regularization](https://mlsanity.com/paper/1810.11654/) (2018)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks](https://mlsanity.com/paper/1803.02579/) (2018)
- [A review: Deep learning for medical image segmentation using multi-modality fusion](https://mlsanity.com/paper/2004.10664/) (2020)
- [Ensembles of Multiple Models and Architectures for Robust Brain Tumour Segmentation](https://mlsanity.com/paper/1711.01468/) (2017)
- [Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge](https://mlsanity.com/paper/1802.10508/) (2018)
- [A deep learning model integrating FCNNs and CRFs for brain tumor segmentation](https://mlsanity.com/paper/1702.04528/) (2017)
- [Automatic Brain Tumor Segmentation using Cascaded Anisotropic Convolutional Neural Networks](https://mlsanity.com/paper/1709.00382/) (2017)
- [Brain tumor segmentation with missing modalities via latent multi-source correlation representation](https://mlsanity.com/paper/2003.08870/) (2020)
- [Joint Sequence Learning and Cross-Modality Convolution for 3D Biomedical Segmentation](https://mlsanity.com/paper/1704.07754/) (2017)
- [Deep CNN ensembles and suggestive annotations for infant brain MRI segmentation](https://mlsanity.com/paper/1712.05319/) (2017)
- [Hierarchical multi-class segmentation of glioma images using networks with multi-level activation function](https://mlsanity.com/paper/1810.09488/) (2018)

## Built on by (most cited first, 10 of 10)

- [Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review](https://mlsanity.com/paper/2307.13125/) (2023)
- [Application of belief functions to medical image segmentation: A review](https://mlsanity.com/paper/2205.01733/) (2022)
- [Lesion-Aware Cross-Phase Attention Network for Renal Tumor Subtype Classification on Multi-Phase CT Scans](https://mlsanity.com/paper/2406.16322/) (2024)
- [Deep evidential fusion with uncertainty quantification and contextual discounting for multimodal medical image segmentation](https://mlsanity.com/paper/2309.05919/) (2023)
- [Enhancing Incomplete Multi-modal Brain Tumor Segmentation with Intra-modal Asymmetry and Inter-modal Dependency](https://mlsanity.com/paper/2406.10175/) (2024)
- [A Performance-Consistent and Computation-Efficient CNN System for High-Quality Automated Brain Tumor Segmentation](https://mlsanity.com/paper/2205.01239/) (2022)
- [A Novel SLCA-UNet Architecture for Automatic MRI Brain Tumor Segmentation](https://mlsanity.com/paper/2307.08048/) (2023)
- [Prototype-Driven and Multi-Expert Integrated Multi-Modal MR Brain Tumor Image Segmentation](https://mlsanity.com/paper/2307.12180/) (2023)
- [Complementary Information Mutual Learning for Multimodality Medical Image Segmentation](https://mlsanity.com/paper/2401.02717/) (2024)
- [Selective Complementary Feature Fusion and Modal Feature Compression Interaction for Brain Tumor Segmentation](https://mlsanity.com/paper/2503.16149/) (2025)

## Abstract (arXiv)

In the field of multimodal segmentation, the correlation between different modalities can be considered for improving the segmentation results. Considering the correlation between different MR modalities, in this paper, we propose a multi-modality segmentation network guided by a novel tri-attention fusion. Our network includes N model-independent encoding paths with N image sources, a tri-attention fusion block, a dual-attention fusion block, and a decoding path. The model independent encoding paths can capture modality-specific features from the N modalities. Considering that not all the features extracted from the encoders are useful for segmentation, we propose to use dual attention based fusion to re-weight the features along the modality and space paths, which can suppress less informative features and emphasize the useful ones for each modality at different positions. Since there exists a strong correlation between different modalities, based on the dual attention fusion block, we propose a correlation attention module to form the tri-attention fusion block. In the correlation attention module, a correlation description block is first used to learn the correlation between modalities and then a constraint based on the correlation is used to guide the network to learn the latent correlated features which are more relevant for segmentation. Finally, the obtained fused feature representation is projected by the decoder to obtain the segmentation results. Our experiment results tested on BraTS 2018 dataset for brain tumor segmentation demonstrate the effectiveness of our proposed method.