# ACN: Adversarial Co-training Network for Brain Tumor Segmentation with Missing Modalities

arXiv: https://arxiv.org/abs/2106.14591 · HTML: https://mlsanity.com/paper/2106.14591/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 28 Jun 2021.

## Builds on (most cited first, 20 of 20)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [3D MRI brain tumor segmentation using autoencoder regularization](https://mlsanity.com/paper/1810.11654/) (2018)
- [Variational Information Distillation for Knowledge Transfer](https://mlsanity.com/paper/1904.05835/) (2019)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision](https://mlsanity.com/paper/2004.07703/) (2020)
- [Brain Tumor Segmentation with Deep Neural Networks](https://mlsanity.com/paper/1505.03540/) (2015)
- [HeMIS: Hetero-Modal Image Segmentation](https://mlsanity.com/paper/1607.05194/) (2016)
- [Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Network](https://mlsanity.com/paper/1904.12200/) (2019)
- [DADA: Depth-aware Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1904.01886/) (2019)
- [A review: Deep learning for medical image segmentation using multi-modality fusion](https://mlsanity.com/paper/2004.10664/) (2020)
- [Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentation](https://mlsanity.com/paper/1907.11150/) (2019)
- [Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion](https://mlsanity.com/paper/2002.09708/) (2020)
- [Knowledge distillation from multi-modal to mono-modal segmentation networks](https://mlsanity.com/paper/2106.09564/) (2021)
- [Cascaded V-Net using ROI masks for brain tumor segmentation](https://mlsanity.com/paper/1812.11588/) (2018)
- [Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration](https://mlsanity.com/paper/2007.02790/) (2020)
- [Modality-Pairing Learning for Brain Tumor Segmentation](https://mlsanity.com/paper/2010.09277/) (2020)
- [Neural Architecture Search for Compressed Sensing Magnetic Resonance Image Reconstruction](https://mlsanity.com/paper/2002.09625/) (2020)
- [Unsupervised Multimodal Image Registration with Adaptative Gradient Guidance](https://mlsanity.com/paper/2011.06216/) (2020)

## Built on by (most cited first, 30 of 43)

- [Multi-modal Learning with Missing Modality via Shared-Specific Feature Modelling](https://mlsanity.com/paper/2307.14126/) (2023)
- [mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentation](https://mlsanity.com/paper/2206.02425/) (2022)
- [Deep Multimodal Learning with Missing Modality: A Survey](https://mlsanity.com/paper/2409.07825/) (2024)
- [M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2303.05302/) (2023)
- [Learnable Cross-modal Knowledge Distillation for Multi-modal Learning with Missing Modality](https://mlsanity.com/paper/2310.01035/) (2023)
- [MMANet: Margin-aware Distillation and Modality-aware Regularization for Incomplete Multimodal Learning](https://mlsanity.com/paper/2304.08028/) (2023)
- [SMU-Net: Style matching U-Net for brain tumor segmentation with missing modalities](https://mlsanity.com/paper/2204.02961/) (2022)
- [Medical Image Segmentation on MRI Images with Missing Modalities: A Review](https://mlsanity.com/paper/2203.06217/) (2022)
- [Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality Benchmarks](https://mlsanity.com/paper/2312.16244/) (2023)
- [PASSION: Towards Effective Incomplete Multi-Modal Medical Image Segmentation with Imbalanced Missing Rates](https://mlsanity.com/paper/2407.14796/) (2024)
- [SFusion: Self-attention based N-to-One Multimodal Fusion Block](https://mlsanity.com/paper/2208.12776/) (2022)
- [Enhancing Modality-Agnostic Representations via Meta-Learning for Brain Tumor Segmentation](https://mlsanity.com/paper/2302.04308/) (2023)
- [Gradient-Guided Modality Decoupling for Missing-Modality Robustness](https://mlsanity.com/paper/2402.16318/) (2024)
- [Multimodal Data Integration for Precision Oncology: Challenges and Future Directions](https://mlsanity.com/paper/2406.19611/) (2024)
- [Federated Modality-specific Encoders and Multimodal Anchors for Personalized Brain Tumor Segmentation](https://mlsanity.com/paper/2403.11803/) (2024)
- [Meta-Learned Modality-Weighted Knowledge Distillation for Robust Multi-Modal Learning with Missing Data](https://mlsanity.com/paper/2405.07155/) (2024)
- [Feasibility of Federated Learning from Client Databases with Different Brain Diseases and MRI Modalities](https://mlsanity.com/paper/2406.11636/) (2024)
- [DC-Seg: Disentangled Contrastive Learning for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2505.11921/) (2025)
- [Deep Learning in Single-Cell Analysis](https://mlsanity.com/paper/2210.12385/) (2022)
- [Dealing with All-stage Missing Modality: Towards A Universal Model with Robust Reconstruction and Personalization](https://mlsanity.com/paper/2406.01987/) (2024)
- [MedMAP: Promoting Incomplete Multi-modal Brain Tumor Segmentation with Alignment](https://mlsanity.com/paper/2408.09465/) (2024)
- [No Modality Left Behind: Adapting to Missing Modalities via Knowledge Distillation for Brain Tumor Segmentation](https://mlsanity.com/paper/2509.15017/) (2025)
- [Towards a Universal 3D Medical Multi-modality Generalization via Learning Personalized Invariant Representation](https://mlsanity.com/paper/2411.06106/) (2024)
- [Multimodal Masked Autoencoder Pre-training for 3D MRI-Based Brain Tumor Analysis with Missing Modalities](https://mlsanity.com/paper/2505.00568/) (2025)
- [Federated Modality-specific Encoders and Partially Personalized Fusion Decoder for Multimodal Brain Tumor Segmentation](https://mlsanity.com/paper/2603.04887/) (2026)
- [RECIST Weakly Supervised Lesion Segmentation via Label-Space Co-Training](https://mlsanity.com/paper/2303.00205/) (2023)
- [Anatomical Consistency Distillation and Inconsistency Synthesis for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2408.13733/) (2024)
- [A Multimodal Feature Distillation with Mamba-Transformer Network for Brain Tumor Segmentation with Incomplete Modalities](https://mlsanity.com/paper/2404.14019/) (2024)
- [Enhancing Incomplete Multi-modal Brain Tumor Segmentation with Intra-modal Asymmetry and Inter-modal Dependency](https://mlsanity.com/paper/2406.10175/) (2024)
- [CCSD: Cross-Modal Compositional Self-Distillation for Robust Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2511.14599/) (2025)

## Abstract (arXiv)

Accurate segmentation of brain tumors from magnetic resonance imaging (MRI) is clinically relevant in diagnoses, prognoses and surgery treatment, which requires multiple modalities to provide complementary morphological and physiopathologic information. However, missing modality commonly occurs due to image corruption, artifacts, different acquisition protocols or allergies to certain contrast agents in clinical practice. Though existing efforts demonstrate the possibility of a unified model for all missing situations, most of them perform poorly when more than one modality is missing. In this paper, we propose a novel Adversarial Co-training Network (ACN) to solve this issue, in which a series of independent yet related models are trained dedicated to each missing situation with significantly better results. Specifically, ACN adopts a novel co-training network, which enables a coupled learning process for both full modality and missing modality to supplement each other's domain and feature representations, and more importantly, to recover the `missing' information of absent modalities. Then, two unsupervised modules, i.e., entropy and knowledge adversarial learning modules are proposed to minimize the domain gap while enhancing prediction reliability and encouraging the alignment of latent representations, respectively. We also adapt modality-mutual information knowledge transfer learning to ACN to retain the rich mutual information among modalities. Extensive experiments on BraTS2018 dataset show that our proposed method significantly outperforms all state-of-the-art methods under any missing situation.