# Joint Learning of Brain Lesion and Anatomy Segmentation from Heterogeneous Datasets

arXiv: https://arxiv.org/abs/1903.03445 · HTML: https://mlsanity.com/paper/1903.03445/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Mar 2019.

## Built on by (most cited first, 5 of 5)

- [MultiTalent: A Multi-Dataset Approach to Medical Image Segmentation](https://mlsanity.com/paper/2303.14444/) (2023)
- [CheXmask: a large-scale dataset of anatomical segmentation masks for multi-center chest x-ray images](https://mlsanity.com/paper/2307.03293/) (2023)
- [Label-set Loss Functions for Partial Supervision: Application to Fetal Brain 3D MRI Parcellation](https://mlsanity.com/paper/2107.03846/) (2021)
- [Learning from Partially Overlapping Labels: Image Segmentation under Annotation Shift](https://mlsanity.com/paper/2107.05938/) (2021)
- [ImputeECG: Deep Learning Reconstruction of Complete 12-Lead Electrocardiograms from Incomplete Recordings for Cardiac Assessment](https://mlsanity.com/paper/2607.05009/) (2026)

## Abstract (arXiv)

Brain lesion and anatomy segmentation in magnetic resonance images are fundamental tasks in neuroimaging research and clinical practice. Given enough training data, convolutional neuronal networks (CNN) proved to outperform all existent techniques in both tasks independently. However, to date, little work has been done regarding simultaneous learning of brain lesion and anatomy segmentation from disjoint datasets. In this work we focus on training a single CNN model to predict brain tissue and lesion segmentations using heterogeneous datasets labeled independently, according to only one of these tasks (a common scenario when using publicly available datasets). We show that label contradiction issues can arise in this case, and propose a novel adaptive cross entropy (ACE) loss function that makes such training possible. We provide quantitative evaluation in two different scenarios, benchmarking the proposed method in comparison with a multi-network approach. Our experiments suggest that ACE loss enables training of single models when standard cross entropy and Dice loss functions tend to fail. Moreover, we show that it is possible to achieve competitive results when comparing with multiple networks trained for independent tasks.