# Rethinking Evaluation of Multiple Sclerosis (MS) Lesion Segmentation Models

arXiv: https://arxiv.org/abs/2605.09666 · HTML: https://mlsanity.com/paper/2605.09666/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 60 references that are in the map.
- First posted on arXiv 10 May 2026.

## Builds on (most cited first, 21 of 21)

- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [3D MRI brain tumor segmentation using autoencoder regularization](https://mlsanity.com/paper/1810.11654/) (2018)
- [nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation](https://mlsanity.com/paper/1809.10486/) (2018)
- [Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach](https://mlsanity.com/paper/1702.04869/) (2017)
- [A Contrast-Adaptive Method for Simultaneous Whole-Brain and Lesion Segmentation in Multiple Sclerosis](https://mlsanity.com/paper/2005.05135/) (2020)
- [Multi-branch Convolutional Neural Network for Multiple Sclerosis Lesion Segmentation](https://mlsanity.com/paper/1811.02942/) (2018)
- [Spatio-temporal Learning from Longitudinal Data for Multiple Sclerosis Lesion Segmentation](https://mlsanity.com/paper/2004.03675/) (2020)
- [Multiple Sclerosis Lesion Analysis in Brain Magnetic Resonance Images: Techniques and Clinical Applications](https://mlsanity.com/paper/2104.10029/) (2021)
- [RSANet: Recurrent Slice-wise Attention Network for Multiple Sclerosis Lesion Segmentation](https://mlsanity.com/paper/2002.12470/) (2020)
- [An Open-Source Tool for Longitudinal Whole-Brain and White Matter Lesion Segmentation](https://mlsanity.com/paper/2207.04534/) (2022)
- [CoactSeg: Learning from Heterogeneous Data for New Multiple Sclerosis Lesion Segmentation](https://mlsanity.com/paper/2307.04513/) (2023)
- [Boosting multiple sclerosis lesion segmentation through attention mechanism](https://mlsanity.com/paper/2304.10790/) (2023)
- [Longitudinal Segmentation of MS Lesions via Temporal Difference Weighting](https://mlsanity.com/paper/2409.13416/) (2024)
- [Towards an accurate and generalizable multiple sclerosis lesion segmentation model using self-ensembled lesion fusion](https://mlsanity.com/paper/2312.01460/) (2023)
- [Shallow vs deep learning architectures for white matter lesion segmentation in the early stages of multiple sclerosis](https://mlsanity.com/paper/1809.03185/) (2018)
- [ICPR 2024 Competition on Multiple Sclerosis Lesion Segmentation -- Methods and Results](https://mlsanity.com/paper/2410.07924/) (2024)
- [Multi-Scale Convolutional-Stack Aggregation for Robust White Matter Hyperintensities Segmentation](https://mlsanity.com/paper/1807.05153/) (2018)
- [Simultaneous lesion and neuroanatomy segmentation in Multiple Sclerosis using deep neural networks](https://mlsanity.com/paper/1901.07419/) (2019)
- [SegHeD: Segmentation of Heterogeneous Data for Multiple Sclerosis Lesions with Anatomical Constraints](https://mlsanity.com/paper/2410.01766/) (2024)
- [Multiple Sclerosis Lesions Segmentation using Attention-Based CNNs in FLAIR Images](https://mlsanity.com/paper/2201.01832/) (2022)
- [SegHeD+: Segmentation of Heterogeneous Data for Multiple Sclerosis Lesions with Anatomical Constraints and Lesion-aware Augmentation](https://mlsanity.com/paper/2412.10946/) (2024)

## Abstract (arXiv)

Multiple Sclerosis (MS) is a chronic autoimmune disease that can significantly reduce the quality of life of a patient. Existing treatment options can only help slow down the progression of the disease. Therefore, early detection and precise monitoring of disease progression are important. Deep learning offers state-of-the-art models for detecting and segmenting MS lesions in brain MRI scans. However, most of these models are evaluated using the Dice score, without accounting for lesion-wise detection and segmentation performance or other metrics that quantify model performance in cases that are complex or confusing for human annotators, or in cases that are essential for disease detection and progression monitoring. In this paper, we highlight the need to rethink the evaluation of MS lesion segmentation models. In this context, we first present problem fingerprinting in detail to highlight what neurologists look for in brain MRI scans for MS detection and progression monitoring, and which metrics are required to properly quantify model performance in these contexts. Additionally, we present an analysis of state-of-the-art models on two open-source datasets using these metrics to highlight their usability for real-world deployment in hospitals.