# Cross-Modality Deep Feature Learning for Brain Tumor Segmentation

arXiv: https://arxiv.org/abs/2201.02356 · HTML: https://mlsanity.com/paper/2201.02356/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 44 references that are in the map.
- First posted on arXiv 7 Jan 2022.

## Builds on (most cited first, 17 of 17)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [On the Efficacy of Knowledge Distillation](https://mlsanity.com/paper/1910.01348/) (2019)
- [3D MRI brain tumor segmentation using autoencoder regularization](https://mlsanity.com/paper/1810.11654/) (2018)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [Recalibrating Fully Convolutional Networks with Spatial and Channel 'Squeeze & Excitation' Blocks](https://mlsanity.com/paper/1808.08127/) (2018)
- [No New-Net](https://mlsanity.com/paper/1809.10483/) (2018)
- [Ensembles of Multiple Models and Architectures for Robust Brain Tumour Segmentation](https://mlsanity.com/paper/1711.01468/) (2017)
- [Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge](https://mlsanity.com/paper/1802.10508/) (2018)
- [Perceptual Adversarial Networks for Image-to-Image Transformation](https://mlsanity.com/paper/1706.09138/) (2017)
- [On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task](https://mlsanity.com/paper/1707.01992/) (2017)
- [Automatic Brain Tumor Segmentation using Cascaded Anisotropic Convolutional Neural Networks](https://mlsanity.com/paper/1709.00382/) (2017)
- [Adaptive feature recombination and recalibration for semantic segmentation with Fully Convolutional Networks](https://mlsanity.com/paper/2006.11193/) (2020)
- [Conditional Adversarial Network for Semantic Segmentation of Brain Tumor](https://mlsanity.com/paper/1708.05227/) (2017)
- [Scalable multimodal convolutional networks for brain tumour segmentation](https://mlsanity.com/paper/1706.08124/) (2017)
- [Global Planar Convolutions for improved context aggregation in Brain Tumor Segmentation](https://mlsanity.com/paper/1908.10281/) (2019)

## Built on by (most cited first, 22 of 22)

- [Deep Industrial Image Anomaly Detection: A Survey](https://mlsanity.com/paper/2301.11514/) (2023)
- [CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer with Modality-Correlated Cross-Attention for Brain Tumor Segmentation](https://mlsanity.com/paper/2207.07370/) (2022)
- [Cross-Modality High-Frequency Transformer for MR Image Super-Resolution](https://mlsanity.com/paper/2203.15314/) (2022)
- [Multimodal Data Integration for Precision Oncology: Challenges and Future Directions](https://mlsanity.com/paper/2406.19611/) (2024)
- [Unsupervised Tissue Segmentation via Deep Constrained Gaussian Network](https://mlsanity.com/paper/2208.02912/) (2022)
- [Revisiting Long-tailed Image Classification: Survey and Benchmarks with New Evaluation Metrics](https://mlsanity.com/paper/2302.01507/) (2023)
- [3D Brain and Heart Volume Generative Models: A Survey](https://mlsanity.com/paper/2210.05952/) (2022)
- [Deep learning automates bidimensional and volumetric tumor burden measurement from MRI in pre- and post-operative glioblastoma patients](https://mlsanity.com/paper/2209.01402/) (2022)
- [MSCoTDet: Language-driven Multi-modal Fusion for Improved Multispectral Pedestrian Detection](https://mlsanity.com/paper/2403.15209/) (2024)
- [Multi-modal Evidential Fusion Network for Trustworthy PET/CT Tumor Segmentation](https://mlsanity.com/paper/2406.18327/) (2024)
- [CCSD: Cross-Modal Compositional Self-Distillation for Robust Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2511.14599/) (2025)
- [A Novel Framework for Brain Tumor Detection Based on Convolutional Variational Generative Models](https://mlsanity.com/paper/2202.09850/) (2022)
- [Uncertainty-Aware Multi-Parametric Magnetic Resonance Image Information Fusion for 3D Object Segmentation](https://mlsanity.com/paper/2211.08783/) (2022)
- [Two-stage MR Image Segmentation Method for Brain Tumors based on Attention Mechanism](https://mlsanity.com/paper/2304.08072/) (2023)
- [Prototype-Driven and Multi-Expert Integrated Multi-Modal MR Brain Tumor Image Segmentation](https://mlsanity.com/paper/2307.12180/) (2023)
- [RIDGE: Reproducibility, Integrity, Dependability, Generalizability, and Efficiency Assessment of Medical Image Segmentation Models](https://mlsanity.com/paper/2401.08847/) (2024)
- [HyCTAS: Multi-Objective Hybrid Convolution-Transformer Architecture Search for Real-Time Image Segmentation](https://mlsanity.com/paper/2403.10413/) (2024)
- [Dual Mutual Learning Network with Global-local Awareness for RGB-D Salient Object Detection](https://mlsanity.com/paper/2501.01648/) (2025)
- [sDREAMER: Self-distilled Mixture-of-Modality-Experts Transformer for Automatic Sleep Staging](https://mlsanity.com/paper/2501.16329/) (2025)
- [Selective Complementary Feature Fusion and Modal Feature Compression Interaction for Brain Tumor Segmentation](https://mlsanity.com/paper/2503.16149/) (2025)
- [GANet-Seg: Adversarial Learning for Brain Tumor Segmentation with Hybrid Generative Models](https://mlsanity.com/paper/2506.21245/) (2025)
- [Quasi-multimodal-based pathophysiological feature learning for retinal disease diagnosis](https://mlsanity.com/paper/2602.03622/) (2026)

## Abstract (arXiv)

Recent advances in machine learning and prevalence of digital medical images have opened up an opportunity to address the challenging brain tumor segmentation (BTS) task by using deep convolutional neural networks. However, different from the RGB image data that are very widespread, the medical image data used in brain tumor segmentation are relatively scarce in terms of the data scale but contain the richer information in terms of the modality property. To this end, this paper proposes a novel cross-modality deep feature learning framework to segment brain tumors from the multi-modality MRI data. The core idea is to mine rich patterns across the multi-modality data to make up for the insufficient data scale. The proposed cross-modality deep feature learning framework consists of two learning processes: the cross-modality feature transition (CMFT) process and the cross-modality feature fusion (CMFF) process, which aims at learning rich feature representations by transiting knowledge across different modality data and fusing knowledge from different modality data, respectively. Comprehensive experiments are conducted on the BraTS benchmarks, which show that the proposed cross-modality deep feature learning framework can effectively improve the brain tumor segmentation performance when compared with the baseline methods and state-of-the-art methods.