# SMU-Net: Style matching U-Net for brain tumor segmentation with missing modalities

arXiv: https://arxiv.org/abs/2204.02961 · HTML: https://mlsanity.com/paper/2204.02961/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 34 references that are in the map.
- First posted on arXiv 6 Apr 2022.

## Builds on (most cited first, 23 of 23)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [Deep Co-Training for Semi-Supervised Image Recognition](https://mlsanity.com/paper/1803.05984/) (2018)
- [HeMIS: Hetero-Modal Image Segmentation](https://mlsanity.com/paper/1607.05194/) (2016)
- [Bi-Directional ConvLSTM U-Net with Densley Connected Convolutions](https://mlsanity.com/paper/1909.00166/) (2019)
- [Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentation](https://mlsanity.com/paper/1907.11150/) (2019)
- [ACN: Adversarial Co-training Network for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2106.14591/) (2021)
- [Latent Correlation Representation Learning for Brain Tumor Segmentation with Missing MRI Modalities](https://mlsanity.com/paper/2104.06231/) (2021)
- [On the Texture Bias for Few-Shot CNN Segmentation](https://mlsanity.com/paper/2003.04052/) (2020)
- [Semi-supervised few-shot learning for medical image segmentation](https://mlsanity.com/paper/2003.08462/) (2020)
- [Contextual Attention Network: Transformer Meets U-Net](https://mlsanity.com/paper/2203.01932/) (2022)
- [Multi-level Context Gating of Embedded Collective Knowledge for Medical Image Segmentation](https://mlsanity.com/paper/2003.05056/) (2020)
- [Neural Style Transfer Improves 3D Cardiovascular MR Image Segmentation on Inconsistent Data](https://mlsanity.com/paper/1909.09716/) (2019)
- [Medical Image Segmentation on MRI Images with Missing Modalities: A Review](https://mlsanity.com/paper/2203.06217/) (2022)
- [Multi-scale Regional Attention Deeplab3+: Multiple Myeloma Plasma Cells Segmentation in Microscopic Images](https://mlsanity.com/paper/2105.06238/) (2021)
- [MR-Contrast-Aware Image-to-Image Translations with Generative Adversarial Networks](https://mlsanity.com/paper/2104.01449/) (2021)
- [Self-Supervised Generative Style Transfer for One-Shot Medical Image Segmentation](https://mlsanity.com/paper/2110.02117/) (2021)
- [Style Separation and Synthesis via Generative Adversarial Networks](https://mlsanity.com/paper/1811.02740/) (2018)
- [HAD-Net: A Hierarchical Adversarial Knowledge Distillation Network for Improved Enhanced Tumour Segmentation Without Post-Contrast Images](https://mlsanity.com/paper/2103.16617/) (2021)
- [Cross-Modal Information Maximization for Medical Imaging: CMIM](https://mlsanity.com/paper/2010.10593/) (2020)
- [Conditional generator and multi-sourcecorrelation guided brain tumor segmentation with missing MR modalities](https://mlsanity.com/paper/2105.13013/) (2021)

## Built on by (most cited first, 30 of 31)

- [Medical Image Segmentation Review: The success of U-Net](https://mlsanity.com/paper/2211.14830/) (2022)
- [HiFormer: Hierarchical Multi-scale Representations Using Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2207.08518/) (2022)
- [Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review](https://mlsanity.com/paper/2301.03505/) (2023)
- [M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2303.05302/) (2023)
- [TransDeepLab: Convolution-Free Transformer-based DeepLab v3+ for Medical Image Segmentation](https://mlsanity.com/paper/2208.00713/) (2022)
- [TransNorm: Transformer Provides a Strong Spatial Normalization Mechanism for a Deep Segmentation Model](https://mlsanity.com/paper/2207.13415/) (2022)
- [Missing Modality Robustness in Semi-Supervised Multi-Modal Semantic Segmentation](https://mlsanity.com/paper/2304.10756/) (2023)
- [PASSION: Towards Effective Incomplete Multi-Modal Medical Image Segmentation with Imbalanced Missing Rates](https://mlsanity.com/paper/2407.14796/) (2024)
- [Enhancing Modality-Agnostic Representations via Meta-Learning for Brain Tumor Segmentation](https://mlsanity.com/paper/2302.04308/) (2023)
- [Multimodal Data Integration for Precision Oncology: Challenges and Future Directions](https://mlsanity.com/paper/2406.19611/) (2024)
- [Federated Modality-specific Encoders and Multimodal Anchors for Personalized Brain Tumor Segmentation](https://mlsanity.com/paper/2403.11803/) (2024)
- [Meta-Learned Modality-Weighted Knowledge Distillation for Robust Multi-Modal Learning with Missing Data](https://mlsanity.com/paper/2405.07155/) (2024)
- [DC-Seg: Disentangled Contrastive Learning for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2505.11921/) (2025)
- [Learning A Robust RGB-Thermal Detector for Extreme Modality Imbalance](https://mlsanity.com/paper/2505.22154/) (2025)
- [Dealing with All-stage Missing Modality: Towards A Universal Model with Robust Reconstruction and Personalization](https://mlsanity.com/paper/2406.01987/) (2024)
- [MedMAP: Promoting Incomplete Multi-modal Brain Tumor Segmentation with Alignment](https://mlsanity.com/paper/2408.09465/) (2024)
- [CoNeS: Conditional neural fields with shift modulation for multi-sequence MRI translation](https://mlsanity.com/paper/2309.03320/) (2023)
- [Multimodal Masked Autoencoder Pre-training for 3D MRI-Based Brain Tumor Analysis with Missing Modalities](https://mlsanity.com/paper/2505.00568/) (2025)
- [A Multimodal Feature Distillation with Mamba-Transformer Network for Brain Tumor Segmentation with Incomplete Modalities](https://mlsanity.com/paper/2404.14019/) (2024)
- [Robust Divergence Learning for Missing-Modality Segmentation](https://mlsanity.com/paper/2411.08305/) (2024)
- [Modality-Agnostic Input Channels Enable Segmentation of Brain lesions in Multimodal MRI with Sequences Unavailable During Training](https://mlsanity.com/paper/2509.09290/) (2025)
- [CCSD: Cross-Modal Compositional Self-Distillation for Robust Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2511.14599/) (2025)
- [Squeeze Excitation Embedded Attention UNet for Brain Tumor Segmentation](https://mlsanity.com/paper/2305.07850/) (2023)
- [Prototype-Driven and Multi-Expert Integrated Multi-Modal MR Brain Tumor Image Segmentation](https://mlsanity.com/paper/2307.12180/) (2023)
- [Unveiling Incomplete Modality Brain Tumor Segmentation: Leveraging Masked Predicted Auto-Encoder and Divergence Learning](https://mlsanity.com/paper/2406.08634/) (2024)
- [Robust Brain Tumor Segmentation with Incomplete MRI Modalities Using H"older Divergence and Mutual Information-Enhanced Knowledge Transfer](https://mlsanity.com/paper/2507.01254/) (2025)
- [AMGFormer: Adaptive Multi-Granular Transformer for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2601.19349/) (2026)
- [Multicentric thrombus segmentation using an attention-based recurrent network with gradual modality dropout](https://mlsanity.com/paper/2604.00817/) (2026)
- [CausalDisenSeg: A Causality-Guided Disentanglement Framework with Counterfactual Reasoning for Robust Brain Tumor Segmentation Under Missing Modalities](https://mlsanity.com/paper/2604.13409/) (2026)
- [Uni-Encoder Meets Multi-Encoders: Representation Before Fusion for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2604.22177/) (2026)

## Abstract (arXiv)

Gliomas are one of the most prevalent types of primary brain tumours, accounting for more than 30% of all cases and they develop from the glial stem or progenitor cells. In theory, the majority of brain tumours could well be identified exclusively by the use of Magnetic Resonance Imaging (MRI). Each MRI modality delivers distinct information on the soft tissue of the human brain and integrating all of them would provide comprehensive data for the accurate segmentation of the glioma, which is crucial for the patient's prognosis, diagnosis, and determining the best follow-up treatment. Unfortunately, MRI is prone to artifacts for a variety of reasons, which might result in missing one or more MRI modalities. Various strategies have been proposed over the years to synthesize the missing modality or compensate for the influence it has on automated segmentation models. However, these methods usually fail to model the underlying missing information. In this paper, we propose a style matching U-Net (SMU-Net) for brain tumour segmentation on MRI images. Our co-training approach utilizes a content and style-matching mechanism to distill the informative features from the full-modality network into a missing modality network. To do so, we encode both full-modality and missing-modality data into a latent space, then we decompose the representation space into a style and content representation. Our style matching module adaptively recalibrates the representation space by learning a matching function to transfer the informative and textural features from a full-modality path into a missing-modality path. Moreover, by modelling the mutual information, our content module surpasses the less informative features and re-calibrates the representation space based on discriminative semantic features. The evaluation process on the BraTS 2018 dataset shows a significant results.