# A Learning Strategy for Contrast-agnostic MRI Segmentation

arXiv: https://arxiv.org/abs/2003.01995 · HTML: https://mlsanity.com/paper/2003.01995/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2020.

## Built on by (most cited first, 30 of 35)

- [SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining](https://mlsanity.com/paper/2107.09559/) (2021)
- [SynthStrip: Skull-Stripping for Any Brain Image](https://mlsanity.com/paper/2203.09974/) (2022)
- [Causality-inspired Single-source Domain Generalization for Medical Image Segmentation](https://mlsanity.com/paper/2111.12525/) (2021)
- [UniverSeg: Universal Medical Image Segmentation](https://mlsanity.com/paper/2304.06131/) (2023)
- [Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain MRI datasets](https://mlsanity.com/paper/2209.02032/) (2022)
- [Source-Free Domain Adaptation for Image Segmentation](https://mlsanity.com/paper/2108.03152/) (2021)
- [MedSyn: Text-guided Anatomy-aware Synthesis of High-Fidelity 3D CT Images](https://mlsanity.com/paper/2310.03559/) (2023)
- [A Robust and Interpretable Deep Learning Framework for Multi-modal Registration via Keypoints](https://mlsanity.com/paper/2304.09941/) (2023)
- [Neuralizer: General Neuroimage Analysis without Re-Training](https://mlsanity.com/paper/2305.02644/) (2023)
- [Anatomy-aware and acquisition-agnostic joint registration with SynthMorph](https://mlsanity.com/paper/2301.11329/) (2023)
- [Challenges for machine learning in clinical translation of big data imaging studies](https://mlsanity.com/paper/2107.05630/) (2021)
- [A self-supervised learning strategy for postoperative brain cavity segmentation simulating resections](https://mlsanity.com/paper/2105.11239/) (2021)
- [Comprehensive analysis of synthetic learning applied to neonatal brain MRI segmentation](https://mlsanity.com/paper/2309.05306/) (2023)
- [A Principled Approach to Failure Analysis and Model Repairment: Demonstration in Medical Imaging](https://mlsanity.com/paper/2109.12347/) (2021)
- [Learn to Ignore: Domain Adaptation for Multi-Site MRI Analysis](https://mlsanity.com/paper/2110.06803/) (2021)
- [Whole Brain Segmentation with Full Volume Neural Network](https://mlsanity.com/paper/2110.15601/) (2021)
- [Robust Segmentation of Brain MRI in the Wild with Hierarchical CNNs and no Retraining](https://mlsanity.com/paper/2203.01969/) (2022)
- [Acquisition-invariant brain MRI segmentation with informative uncertainties](https://mlsanity.com/paper/2111.04094/) (2021)
- [Accurate super-resolution low-field brain MRI](https://mlsanity.com/paper/2202.03564/) (2022)
- [Synthetic Low-Field MRI Super-Resolution Via Nested U-Net Architecture](https://mlsanity.com/paper/2211.15047/) (2022)
- [Empirical Analysis of a Segmentation Foundation Model in Prostate Imaging](https://mlsanity.com/paper/2307.03266/) (2023)
- [Harmonization and the Worst Scanner Syndrome](https://mlsanity.com/paper/2101.06255/) (2021)
- [An MRF-UNet Product of Experts for Image Segmentation](https://mlsanity.com/paper/2104.05495/) (2021)
- [Exploring contrast generalisation in deep learning-based brain MRI-to-CT synthesis](https://mlsanity.com/paper/2303.10202/) (2023)
- [Matching in the Wild: Learning Anatomical Embeddings for Multi-Modality Images](https://mlsanity.com/paper/2307.03535/) (2023)
- [FastSurfer-HypVINN: Automated sub-segmentation of the hypothalamus and adjacent structures on high-resolutional brain MRI](https://mlsanity.com/paper/2308.12736/) (2023)
- [Towards contrast-agnostic soft segmentation of the spinal cord](https://mlsanity.com/paper/2310.15402/) (2023)
- [An augmentation strategy to mimic multi-scanner variability in MRI](https://mlsanity.com/paper/2103.12595/) (2021)
- [Neuroverse3D: Developing In-Context Learning Universal Model for Neuroimaging in 3D](https://mlsanity.com/paper/2503.02410/) (2025)
- [Deep infant brain segmentation from multi-contrast MRI](https://mlsanity.com/paper/2512.05114/) (2025)

## Abstract (arXiv)

We present a deep learning strategy that enables, for the first time, contrast-agnostic semantic segmentation of completely unpreprocessed brain MRI scans, without requiring additional training or fine-tuning for new modalities. Classical Bayesian methods address this segmentation problem with unsupervised intensity models, but require significant computational resources. In contrast, learning-based methods can be fast at test time, but are sensitive to the data available at training. Our proposed learning method, SynthSeg, leverages a set of training segmentations (no intensity images required) to generate synthetic sample images of widely varying contrasts on the fly during training. These samples are produced using the generative model of the classical Bayesian segmentation framework, with randomly sampled parameters for appearance, deformation, noise, and bias field. Because each mini-batch has a different synthetic contrast, the final network is not biased towards any MRI contrast. We comprehensively evaluate our approach on four datasets comprising over 1,000 subjects and four types of MR contrast. The results show that our approach successfully segments every contrast in the data, performing slightly better than classical Bayesian segmentation, and three orders of magnitude faster. Moreover, even within the same type of MRI contrast, our strategy generalizes significantly better across datasets, compared to training using real images. Finally, we find that synthesizing a broad range of contrasts, even if unrealistic, increases the generalization of the neural network. Our code and model are open source at https://github.com/BBillot/SynthSeg.