# Learning Cross-Modal Deep Representations for Multi-Modal MR Image Segmentation

arXiv: https://arxiv.org/abs/1908.01997 · HTML: https://mlsanity.com/paper/1908.01997/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2019.

## Built on by (most cited first, 9 of 9)

- [A Novel Hybrid Convolutional Neural Network for Accurate Organ Segmentation in 3D Head and Neck CT Images](https://mlsanity.com/paper/2109.12634/) (2021)
- [SimPLe: Similarity-Aware Propagation Learning for Weakly-Supervised Breast Cancer Segmentation in DCE-MRI](https://mlsanity.com/paper/2306.16714/) (2023)
- [Cross-modality Attention Adapter: A Glioma Segmentation Fine-tuning Method for SAM Using Multimodal Brain MR Images](https://mlsanity.com/paper/2307.01124/) (2023)
- [Prototype Learning Guided Hybrid Network for Breast Tumor Segmentation in DCE-MRI](https://mlsanity.com/paper/2408.05803/) (2024)
- [CNN-based explanation ensembling for dataset, representation and explanations evaluation](https://mlsanity.com/paper/2404.10387/) (2024)
- [Evidential learning driven Breast Tumor Segmentation with Stage-divided Vision-Language Interaction](https://mlsanity.com/paper/2603.11206/) (2026)
- [Uncertainty-Aware Multi-Parametric Magnetic Resonance Image Information Fusion for 3D Object Segmentation](https://mlsanity.com/paper/2211.08783/) (2022)
- [LESEN: Label-Efficient deep learning for Multi-parametric MRI-based Visual Pathway Segmentation](https://mlsanity.com/paper/2401.01654/) (2024)
- [Cross-Sequence Semi-Supervised Learning for Multi-Parametric MRI-Based Visual Pathway Delineation](https://mlsanity.com/paper/2505.19733/) (2025)

## Abstract (arXiv)

Multi-modal magnetic resonance imaging (MRI) is essential in clinics for comprehensive diagnosis and surgical planning. Nevertheless, the segmentation of multi-modal MR images tends to be time-consuming and challenging. Convolutional neural network (CNN)-based multi-modal MR image analysis commonly proceeds with multiple down-sampling streams fused at one or several layers. Although inspiring performance has been achieved, the feature fusion is usually conducted through simple summation or concatenation without optimization. In this work, we propose a supervised image fusion method to selectively fuse the useful information from different modalities and suppress the respective noise signals. Specifically, an attention block is introduced as guidance for the information selection. From the different modalities, one modality that contributes most to the results is selected as the master modality, which supervises the information selection of the other assistant modalities. The effectiveness of the proposed method is confirmed through breast mass segmentation in MR images of two modalities and better segmentation results are achieved compared to the state-of-the-art methods.