# ModDrop++: A Dynamic Filter Network with Intra-subject Co-training for Multiple Sclerosis Lesion Segmentation with Missing Modalities

arXiv: https://arxiv.org/abs/2203.04959 · HTML: https://mlsanity.com/paper/2203.04959/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 7 Mar 2022.

## Builds on (most cited first, 12 of 12)

- [HeMIS: Hetero-Modal Image Segmentation](https://mlsanity.com/paper/1607.05194/) (2016)
- [DoDNet: Learning to segment multi-organ and tumors from multiple partially labeled datasets](https://mlsanity.com/paper/2011.10217/) (2020)
- [Knowledge distillation from multi-modal to mono-modal segmentation networks](https://mlsanity.com/paper/2106.09564/) (2021)
- [Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach](https://mlsanity.com/paper/1702.04869/) (2017)
- [DeepLesionBrain: Towards a broader deep-learning generalization for multiple sclerosis lesion segmentation](https://mlsanity.com/paper/2012.07950/) (2020)
- [A Unified Hyper-GAN Model for Unpaired Multi-contrast MR Image Translation](https://mlsanity.com/paper/2107.11945/) (2021)
- [Scanner Invariant Multiple Sclerosis Lesion Segmentation from MRI](https://mlsanity.com/paper/1910.10035/) (2019)
- [Latent Filter Scaling for Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1812.09877/) (2018)
- [A non-hierarchical attention network with modality dropout for textual response generation in multimodal dialogue systems](https://mlsanity.com/paper/2110.09702/) (2021)
- [A Self-Adaptive Network For Multiple Sclerosis Lesion Segmentation From Multi-Contrast MRI With Various Imaging Protocols](https://mlsanity.com/paper/1811.07491/) (2018)
- [Input Dropout for Spatially Aligned Modalities](https://mlsanity.com/paper/2002.02852/) (2020)
- [Automated Detection of Cortical Lesions in Multiple Sclerosis Patients with 7T MRI](https://mlsanity.com/paper/2008.06780/) (2020)

## Built on by (most cited first, 22 of 22)

- [SimMLM: A Simple Framework for Multi-modal Learning with Missing Modality](https://mlsanity.com/paper/2507.19264/) (2025)
- [Omni-Seg: A Scale-aware Dynamic Network for Renal Pathological Image Segmentation](https://mlsanity.com/paper/2206.13632/) (2022)
- [Enhancing Modality-Agnostic Representations via Meta-Learning for Brain Tumor Segmentation](https://mlsanity.com/paper/2302.04308/) (2023)
- [Distilled Prompt Learning for Incomplete Multimodal Survival Prediction](https://mlsanity.com/paper/2503.01653/) (2025)
- [Multimodal Data Integration for Precision Oncology: Challenges and Future Directions](https://mlsanity.com/paper/2406.19611/) (2024)
- [Learning Site-specific Styles for Multi-institutional Unsupervised Cross-modality Domain Adaptation](https://mlsanity.com/paper/2311.12437/) (2023)
- [Unique MS Lesion Identification from MRI](https://mlsanity.com/paper/2410.09639/) (2024)
- [SSL^2: Self-Supervised Learning meets Semi-Supervised Learning: Multiple Sclerosis Segmentation in 7T-MRI from large-scale 3T-MRI](https://mlsanity.com/paper/2303.05026/) (2023)
- [A Unified Learning Model for Estimating Fiber Orientation Distribution Functions on Heterogeneous Multi-shell Diffusion-weighted MRI](https://mlsanity.com/paper/2303.16376/) (2023)
- [Self Pre-training with Adaptive Mask Autoencoders for Variable-Contrast 3D Medical Imaging](https://mlsanity.com/paper/2501.09096/) (2025)
- [UNISELF: A Unified Network with Instance Normalization and Self-Ensembled Lesion Fusion for Multiple Sclerosis Lesion Segmentation](https://mlsanity.com/paper/2508.03982/) (2025)
- [Multi-Modal Mamba Modeling for Survival Prediction (M4Survive): Adapting Joint Foundation Model Representations](https://mlsanity.com/paper/2503.10057/) (2025)
- [crossMoDA Challenge: Evolution of Cross-Modality Domain Adaptation Techniques for Vestibular Schwannoma and Cochlea Segmentation from 2021 to 2023](https://mlsanity.com/paper/2506.12006/) (2025)
- [Diffusion Autoencoders with Perceivers for Long, Irregular and Multimodal Astronomical Sequences](https://mlsanity.com/paper/2510.20595/) (2025)
- [CATS v2: Hybrid encoders for robust medical segmentation](https://mlsanity.com/paper/2308.06377/) (2023)
- [Automatic segmentation of Organs at Risk in Head and Neck cancer patients from CT and MRI scans](https://mlsanity.com/paper/2405.10833/) (2024)
- [Predicting fluorescent labels in label-free microscopy images with pix2pix and adaptive loss in Light My Cells challenge](https://mlsanity.com/paper/2406.15716/) (2024)
- [AdaViT: Adaptive Vision Transformer for Flexible Pretrain and Finetune with Variable 3D Medical Image Modalities](https://mlsanity.com/paper/2504.03589/) (2025)
- [Multi-Plane Vision Transformer for Hemorrhage Classification Using Axial and Sagittal MRI Data](https://mlsanity.com/paper/2505.07349/) (2025)
- [VIViT: Variable-Input Vision Transformer Framework for 3D MR Image Segmentation](https://mlsanity.com/paper/2505.08693/) (2025)
- [Multicentric thrombus segmentation using an attention-based recurrent network with gradual modality dropout](https://mlsanity.com/paper/2604.00817/) (2026)
- [Sequential Modality Dropout for Robust Multi-Modal Sequential Recommendation](https://mlsanity.com/paper/2608.10240/) (2026)

## Abstract (arXiv)

Multiple Sclerosis (MS) is a chronic neuroinflammatory disease and multi-modality MRIs are routinely used to monitor MS lesions. Many automatic MS lesion segmentation models have been developed and have reached human-level performance. However, most established methods assume the MRI modalities used during training are also available during testing, which is not guaranteed in clinical practice. Previously, a training strategy termed Modality Dropout (ModDrop) has been applied to MS lesion segmentation to achieve the state-of-the-art performance with missing modality. In this paper, we present a novel method dubbed ModDrop++ to train a unified network adaptive to an arbitrary number of input MRI sequences. ModDrop++ upgrades the main idea of ModDrop in two key ways. First, we devise a plug-and-play dynamic head and adopt a filter scaling strategy to improve the expressiveness of the network. Second, we design a co-training strategy to leverage the intra-subject relation between full modality and missing modality. Specifically, the intra-subject co-training strategy aims to guide the dynamic head to generate similar feature representations between the full- and missing-modality data from the same subject. We use two public MS datasets to show the superiority of ModDrop++. Source code and trained models are available at https://github.com/han-liu/ModDropPlusPlus.