# Segmentation of Multiple Sclerosis Lesions across Hospitals: Learn Continually or Train from Scratch?

arXiv: https://arxiv.org/abs/2210.15091 · HTML: https://mlsanity.com/paper/2210.15091/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 13 references that are in the map.
- First posted on arXiv 27 Oct 2022.

## Builds on (most cited first, 8 of 8)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [SoftSeg: Advantages of soft versus binary training for image segmentation](https://mlsanity.com/paper/2011.09041/) (2020)
- [Towards continual learning in medical imaging](https://mlsanity.com/paper/1811.02496/) (2018)
- [Dynamic memory to alleviate catastrophic forgetting in continuous learning settings](https://mlsanity.com/paper/2007.02639/) (2020)
- [Towards continuous learning for glioma segmentation with elastic weight consolidation](https://mlsanity.com/paper/1909.11479/) (2019)

## Built on by (most cited first, 3 of 3)

- [Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospects](https://mlsanity.com/paper/2312.17004/) (2023)
- [Dynamic Robot-Assisted Surgery with Hierarchical Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2508.01713/) (2025)
- [Monitoring morphometric drift in lifelong learning segmentation of the spinal cord](https://mlsanity.com/paper/2505.01364/) (2025)

## Abstract (arXiv)

Segmentation of Multiple Sclerosis (MS) lesions is a challenging problem. Several deep-learning-based methods have been proposed in recent years. However, most methods tend to be static, that is, a single model trained on a large, specialized dataset, which does not generalize well. Instead, the model should learn across datasets arriving sequentially from different hospitals by building upon the characteristics of lesions in a continual manner. In this regard, we explore experience replay, a well-known continual learning method, in the context of MS lesion segmentation across multi-contrast data from 8 different hospitals. Our experiments show that replay is able to achieve positive backward transfer and reduce catastrophic forgetting compared to sequential fine-tuning. Furthermore, replay outperforms the multi-domain training, thereby emerging as a promising solution for the segmentation of MS lesions. The code is available at this link: https://github.com/naga-karthik/continual-learning-ms