# SegHeD: Segmentation of Heterogeneous Data for Multiple Sclerosis Lesions with Anatomical Constraints

arXiv: https://arxiv.org/abs/2410.01766 · HTML: https://mlsanity.com/paper/2410.01766/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Builds on (most cited first, 13 of 13)

- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining](https://mlsanity.com/paper/2107.09559/) (2021)
- [CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection](https://mlsanity.com/paper/2301.00785/) (2023)
- [nnFormer: Interleaved Transformer for Volumetric Segmentation](https://mlsanity.com/paper/2109.03201/) (2021)
- [Anatomical Priors in Convolutional Networks for Unsupervised Biomedical Segmentation](https://mlsanity.com/paper/1903.03148/) (2019)
- [Biomedical image analysis competitions: The state of current participation practice](https://mlsanity.com/paper/2212.08568/) (2022)
- [Self-Supervised Learning for Medical Image Data with Anatomy-Oriented Imaging Planes](https://mlsanity.com/paper/2403.16499/) (2024)
- [Spatio-temporal Learning from Longitudinal Data for Multiple Sclerosis Lesion Segmentation](https://mlsanity.com/paper/2004.03675/) (2020)
- [CoactSeg: Learning from Heterogeneous Data for New Multiple Sclerosis Lesion Segmentation](https://mlsanity.com/paper/2307.04513/) (2023)
- [LesionMix: A Lesion-Level Data Augmentation Method for Medical Image Segmentation](https://mlsanity.com/paper/2308.09026/) (2023)
- [Segmentation of MRI head anatomy using deep volumetric networks and multiple spatial priors](https://mlsanity.com/paper/1905.10010/) (2019)

## Built on by (most cited first, 3 of 3)

- [SegHeD+: Segmentation of Heterogeneous Data for Multiple Sclerosis Lesions with Anatomical Constraints and Lesion-aware Augmentation](https://mlsanity.com/paper/2412.10946/) (2024)
- [Self-Supervised Multi-Stage Domain Unlearning for White-Matter Lesion Segmentation](https://mlsanity.com/paper/2603.13328/) (2026)
- [Rethinking Evaluation of Multiple Sclerosis (MS) Lesion Segmentation Models](https://mlsanity.com/paper/2605.09666/) (2026)

## Abstract (arXiv)

Assessment of lesions and their longitudinal progression from brain magnetic resonance (MR) images plays a crucial role in diagnosing and monitoring multiple sclerosis (MS). Machine learning models have demonstrated a great potential for automated MS lesion segmentation. Training such models typically requires large-scale high-quality datasets that are consistently annotated. However, MS imaging datasets are often small, segregated across multiple sites, with different formats (cross-sectional or longitudinal), and diverse annotation styles. This poses a significant challenge to train a unified MS lesion segmentation model. To tackle this challenge, we present SegHeD, a novel multi-dataset multi-task segmentation model that can incorporate heterogeneous data as input and perform all-lesion, new-lesion, as well as vanishing-lesion segmentation. Furthermore, we account for domain knowledge about MS lesions, incorporating longitudinal, spatial, and volumetric constraints into the segmentation model. SegHeD is assessed on five MS datasets and achieves a high performance in all, new, and vanishing-lesion segmentation, outperforming several state-of-the-art methods in this field.