# Medical Image Segmentation on MRI Images with Missing Modalities: A Review

arXiv: https://arxiv.org/abs/2203.06217 · HTML: https://mlsanity.com/paper/2203.06217/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 103 references that are in the map.
- First posted on arXiv 11 Mar 2022.

## Builds on (most cited first, 30 of 47)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [3D MRI brain tumor segmentation using autoencoder regularization](https://mlsanity.com/paper/1810.11654/) (2018)
- [CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation](https://mlsanity.com/paper/2001.06535/) (2020)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [Multimodal Generative Models for Scalable Weakly-Supervised Learning](https://mlsanity.com/paper/1802.05335/) (2018)
- [ResViT: Residual vision transformers for multi-modal medical image synthesis](https://mlsanity.com/paper/2106.16031/) (2021)
- [HeMIS: Hetero-Modal Image Segmentation](https://mlsanity.com/paper/1607.05194/) (2016)
- [Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Network](https://mlsanity.com/paper/1904.12200/) (2019)
- [Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesis](https://mlsanity.com/paper/2002.05000/) (2020)
- [Bi-Directional ConvLSTM U-Net with Densley Connected Convolutions](https://mlsanity.com/paper/1909.00166/) (2019)
- [Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentation](https://mlsanity.com/paper/1907.11150/) (2019)
- [Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion](https://mlsanity.com/paper/2002.09708/) (2020)
- [Knowledge distillation from multi-modal to mono-modal segmentation networks](https://mlsanity.com/paper/2106.09564/) (2021)
- [No New-Net](https://mlsanity.com/paper/1809.10483/) (2018)
- [Modality-aware Mutual Learning for Multi-modal Medical Image Segmentation](https://mlsanity.com/paper/2107.09842/) (2021)
- [ACN: Adversarial Co-training Network for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2106.14591/) (2021)
- [Brain Tumor Segmentation on MRI with Missing Modalities](https://mlsanity.com/paper/1904.07290/) (2019)
- [Latent Correlation Representation Learning for Brain Tumor Segmentation with Missing MRI Modalities](https://mlsanity.com/paper/2104.06231/) (2021)
- [On the Texture Bias for Few-Shot CNN Segmentation](https://mlsanity.com/paper/2003.04052/) (2020)
- [DiamondGAN: Unified Multi-Modal Generative Adversarial Networks for MRI Sequences Synthesis](https://mlsanity.com/paper/1904.12894/) (2019)
- [A deep learning model integrating FCNNs and CRFs for brain tumor segmentation](https://mlsanity.com/paper/1702.04528/) (2017)
- [Semi-supervised few-shot learning for medical image segmentation](https://mlsanity.com/paper/2003.08462/) (2020)
- [Contextual Attention Network: Transformer Meets U-Net](https://mlsanity.com/paper/2203.01932/) (2022)
- [Multi-level Context Gating of Embedded Collective Knowledge for Medical Image Segmentation](https://mlsanity.com/paper/2003.05056/) (2020)
- [Representation Disentanglement for Multi-modal brain MR Analysis](https://mlsanity.com/paper/2102.11456/) (2021)

## Built on by (most cited first, 18 of 18)

- [Medical Image Segmentation Review: The success of U-Net](https://mlsanity.com/paper/2211.14830/) (2022)
- [Deep Multimodal Learning with Missing Modality: A Survey](https://mlsanity.com/paper/2409.07825/) (2024)
- [SDSTrack: Self-Distillation Symmetric Adapter Learning for Multi-Modal Visual Object Tracking](https://mlsanity.com/paper/2403.16002/) (2024)
- [TransDeepLab: Convolution-Free Transformer-based DeepLab v3+ for Medical Image Segmentation](https://mlsanity.com/paper/2208.00713/) (2022)
- [TransNorm: Transformer Provides a Strong Spatial Normalization Mechanism for a Deep Segmentation Model](https://mlsanity.com/paper/2207.13415/) (2022)
- [SMU-Net: Style matching U-Net for brain tumor segmentation with missing modalities](https://mlsanity.com/paper/2204.02961/) (2022)
- [DAE-Former: Dual Attention-guided Efficient Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2212.13504/) (2022)
- [Multi-Modality Cardiac Image Computing: A Survey](https://mlsanity.com/paper/2208.12881/) (2022)
- [ADAPT: Multimodal Learning for Detecting Physiological Changes under Missing Modalities](https://mlsanity.com/paper/2407.03836/) (2024)
- [Understanding the Tricks of Deep Learning in Medical Image Segmentation: Challenges and Future Directions](https://mlsanity.com/paper/2209.10307/) (2022)
- [Intervertebral Disc Labeling With Learning Shape Information, A Look Once Approach](https://mlsanity.com/paper/2204.02943/) (2022)
- [CoNeS: Conditional neural fields with shift modulation for multi-sequence MRI translation](https://mlsanity.com/paper/2309.03320/) (2023)
- [Anatomical Consistency Distillation and Inconsistency Synthesis for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2408.13733/) (2024)
- [Multi-modal Contrastive Learning for Tumor-specific Missing Modality Synthesis](https://mlsanity.com/paper/2502.19390/) (2025)
- [Efficient Parameter Adaptation for Multi-Modal Medical Image Segmentation and Prognosis](https://mlsanity.com/paper/2504.13645/) (2025)
- [VIViT: Variable-Input Vision Transformer Framework for 3D MR Image Segmentation](https://mlsanity.com/paper/2505.08693/) (2025)
- [Improved mmFormer for Liver Fibrosis Staging via Missing-Modality Compensation](https://mlsanity.com/paper/2509.16436/) (2025)
- [Medical AI Encodes a "Feeling of Error": Verifying Cancer Segmentation via Internal Concepts](https://mlsanity.com/paper/2609.08879/) (2026)

## Abstract (arXiv)

Dealing with missing modalities in Magnetic Resonance Imaging (MRI) and overcoming their negative repercussions is considered a hurdle in biomedical imaging. The combination of a specified set of modalities, which is selected depending on the scenario and anatomical part being scanned, will provide medical practitioners with full information about the region of interest in the human body, hence the missing MRI sequences should be reimbursed. The compensation of the adverse impact of losing useful information owing to the lack of one or more modalities is a well-known challenge in the field of computer vision, particularly for medical image processing tasks including tumour segmentation, tissue classification, and image generation. Various approaches have been developed over time to mitigate this problem's negative implications and this literature review goes through a significant number of the networks that seek to do so. The approaches reviewed in this work are reviewed in detail, including earlier techniques such as synthesis methods as well as later approaches that deploy deep learning, such as common latent space models, knowledge distillation networks, mutual information maximization, and generative adversarial networks (GANs). This work discusses the most important approaches that have been offered at the time of this writing, examining the novelty, strength, and weakness of each one. Furthermore, the most commonly used MRI datasets are highlighted and described. The main goal of this research is to offer a performance evaluation of missing modality compensating networks, as well as to outline future strategies for dealing with this issue.