# PIMMS: Permutation Invariant Multi-Modal Segmentation

arXiv: https://arxiv.org/abs/1807.06537 · HTML: https://mlsanity.com/paper/1807.06537/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2018.

## Built on by (most cited first, 9 of 9)

- [Medical Image Segmentation on MRI Images with Missing Modalities: A Review](https://mlsanity.com/paper/2203.06217/) (2022)
- [Modality Completion via Gaussian Process Prior Variational Autoencoders for Multi-Modal Glioma Segmentation](https://mlsanity.com/paper/2107.03442/) (2021)
- [Unified Brain MR-Ultrasound Synthesis using Multi-Modal Hierarchical Representations](https://mlsanity.com/paper/2309.08747/) (2023)
- [Unified Cross-Modal Medical Image Synthesis with Hierarchical Mixture of Product-of-Experts](https://mlsanity.com/paper/2410.19378/) (2024)
- [MedMAP: Promoting Incomplete Multi-modal Brain Tumor Segmentation with Alignment](https://mlsanity.com/paper/2408.09465/) (2024)
- [Towards a Universal 3D Medical Multi-modality Generalization via Learning Personalized Invariant Representation](https://mlsanity.com/paper/2411.06106/) (2024)
- [FgC2F-UDiff: Frequency-guided and Coarse-to-fine Unified Diffusion Model for Multi-modality Missing MRI Synthesis](https://mlsanity.com/paper/2501.03526/) (2025)
- [FastSurfer-HypVINN: Automated sub-segmentation of the hypothalamus and adjacent structures on high-resolutional brain MRI](https://mlsanity.com/paper/2308.12736/) (2023)
- [Med-K2N: Flexible K-to-N Modality Translation for Medical Image Synthesis](https://mlsanity.com/paper/2510.02815/) (2025)

## Abstract (arXiv)

In a research context, image acquisition will often involve a pre-defined static protocol and the data will be of high quality. If we are to build applications that work in hospitals without significant operational changes in care delivery, algorithms should be designed to cope with the available data in the best possible way. In a clinical environment, imaging protocols are highly flexible, with MRI sequences commonly missing appropriate sequence labeling (e.g. T1, T2, FLAIR). To this end we introduce PIMMS, a Permutation Invariant Multi-Modal Segmentation technique that is able to perform inference over sets of MRI scans without using modality labels. We present results which show that our convolutional neural network can, in some settings, outperform a baseline model which utilizes modality labels, and achieve comparable performance otherwise.