# Modality-Pairing Learning for Brain Tumor Segmentation

arXiv: https://arxiv.org/abs/2010.09277 · HTML: https://mlsanity.com/paper/2010.09277/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2020.

## Built on by (most cited first, 17 of 17)

- [ACN: Adversarial Co-training Network for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2106.14591/) (2021)
- [CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer with Modality-Correlated Cross-Attention for Brain Tumor Segmentation](https://mlsanity.com/paper/2207.07370/) (2022)
- [Uncertainty-aware Multi-modal Learning via Cross-modal Random Network Prediction](https://mlsanity.com/paper/2207.10851/) (2022)
- [Panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps](https://mlsanity.com/paper/2312.02608/) (2023)
- [Cutting-edge 3D Medical Image Segmentation Methods in 2020: Are Happy Families All Alike?](https://mlsanity.com/paper/2101.00232/) (2021)
- [Large-Kernel Attention for 3D Medical Image Segmentation](https://mlsanity.com/paper/2207.11225/) (2022)
- [TranSiam: Fusing Multimodal Visual Features Using Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2204.12185/) (2022)
- [Exploiting Partial Common Information Microstructure for Multi-Modal Brain Tumor Segmentation](https://mlsanity.com/paper/2302.02521/) (2023)
- [Combining CNNs With Transformer for Multimodal 3D MRI Brain Tumor Segmentation With Self-Supervised Pretraining](https://mlsanity.com/paper/2110.07919/) (2021)
- [CASPIANET++: A Multidimensional Channel-Spatial Asymmetric Attention Network with Noisy Student Curriculum Learning Paradigm for Brain Tumor Segmentation](https://mlsanity.com/paper/2107.04099/) (2021)
- [Deep Superpixel Generation and Clustering for Weakly Supervised Segmentation of Brain Tumors in MR Images](https://mlsanity.com/paper/2209.09930/) (2022)
- [Learning to segment anatomy and lesions from disparately labeled sources in brain MRI](https://mlsanity.com/paper/2503.18840/) (2025)
- [Learning Multi-Modal Brain Tumor Segmentation from Privileged Semi-Paired MRI Images with Curriculum Disentanglement Learning](https://mlsanity.com/paper/2208.12781/) (2022)
- [Towards fully automated deep-learning-based brain tumor segmentation: is brain extraction still necessary?](https://mlsanity.com/paper/2212.07497/) (2022)
- [Medical Image Analysis using Deep Relational Learning](https://mlsanity.com/paper/2303.16099/) (2023)
- [SYNAPSE-Net: A Unified Framework with Lesion-Aware Hierarchical Gating for Robust Segmentation of Heterogeneous Brain Lesions](https://mlsanity.com/paper/2510.26961/) (2025)
- [Fairboard: a quantitative framework for equity assessment of healthcare models](https://mlsanity.com/paper/2604.09656/) (2026)

## Abstract (arXiv)

Automatic brain tumor segmentation from multi-modality Magnetic Resonance Images (MRI) using deep learning methods plays an important role in assisting the diagnosis and treatment of brain tumor. However, previous methods mostly ignore the latent relationship among different modalities. In this work, we propose a novel end-to-end Modality-Pairing learning method for brain tumor segmentation. Paralleled branches are designed to exploit different modality features and a series of layer connections are utilized to capture complex relationships and abundant information among modalities. We also use a consistency loss to minimize the prediction variance between two branches. Besides, learning rate warmup strategy is adopted to solve the problem of the training instability and early over-fitting. Lastly, we use average ensemble of multiple models and some post-processing techniques to get final results. Our method is tested on the BraTS 2020 online testing dataset, obtaining promising segmentation performance, with average dice scores of 0.891, 0.842, 0.816 for the whole tumor, tumor core and enhancing tumor, respectively. We won the second place of the BraTS 2020 Challenge for the tumor segmentation task.