# Detailed delineation of the fetal brain in diffusion MRI via multi-task learning

arXiv: https://arxiv.org/abs/2409.06716 · HTML: https://mlsanity.com/paper/2409.06716/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 67 references that are in the map.
- First posted on arXiv 26 Aug 2024.

## Builds on (most cited first, 28 of 28)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Multi-Task Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2009.09796/) (2020)
- [Efficiently Identifying Task Groupings for Multi-Task Learning](https://mlsanity.com/paper/2109.04617/) (2021)
- [Deep Learning is Robust to Massive Label Noise](https://mlsanity.com/paper/1705.10694/) (2017)
- [Instance-aware Semantic Segmentation via Multi-task Network Cascades](https://mlsanity.com/paper/1512.04412/) (2015)
- [Branched Multi-Task Networks: Deciding What Layers To Share](https://mlsanity.com/paper/1904.02920/) (2019)
- [An automatic multi-tissue human fetal brain segmentation benchmark using the Fetal Tissue Annotation Dataset](https://mlsanity.com/paper/2010.15526/) (2020)
- [TractSeg - Fast and accurate white matter tract segmentation](https://mlsanity.com/paper/1805.07103/) (2018)
- [Convolution-Free Medical Image Segmentation using Transformers](https://mlsanity.com/paper/2102.13645/) (2021)
- [Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities](https://mlsanity.com/paper/1704.03379/) (2017)
- [Adaptive Scheduling for Multi-Task Learning](https://mlsanity.com/paper/1909.06434/) (2019)
- [Fetal Brain Tissue Annotation and Segmentation Challenge Results](https://mlsanity.com/paper/2204.09573/) (2022)
- [Automatic brain tissue segmentation in fetal MRI using convolutional neural networks](https://mlsanity.com/paper/1906.04713/) (2019)
- [A Deep Attentive Convolutional Neural Network for Automatic Cortical Plate Segmentation in Fetal MRI](https://mlsanity.com/paper/2004.12847/) (2020)
- [The White Matter Query Language: A Novel Approach for Describing Human White Matter Anatomy](https://mlsanity.com/paper/1512.07404/) (2015)
- [Diffusion MRI with Machine Learning](https://mlsanity.com/paper/2402.00019/) (2024)
- [Tractography and machine learning: Current state and open challenges](https://mlsanity.com/paper/1902.05568/) (2019)
- [Distributionally Robust Segmentation of Abnormal Fetal Brain 3D MRI](https://mlsanity.com/paper/2108.04175/) (2021)
- [Efficient multi-class fetal brain segmentation in high resolution MRI reconstructions with noisy labels](https://mlsanity.com/paper/2009.06275/) (2020)
- [Segmentation of the cortical plate in fetal brain MRI with a topological loss](https://mlsanity.com/paper/2010.12391/) (2020)
- [Learning to segment fetal brain tissue from noisy annotations](https://mlsanity.com/paper/2203.14962/) (2022)
- [Neuro4Neuro: A neural network approach for neural tract segmentation using large-scale population-based diffusion imaging](https://mlsanity.com/paper/2005.12838/) (2020)
- [HAMLET: Hierarchical Harmonic Filters for Learning Tracts from Diffusion MRI](https://mlsanity.com/paper/1807.01068/) (2018)

## Built on by (most cited first, 3 of 3)

- [Atlas-Assisted Segment Anything Model for Fetal Brain MRI (FeTal-SAM)](https://mlsanity.com/paper/2601.15759/) (2026)
- [DeepMultiConnectome: Deep Multi-Task Prediction of Structural Connectomes Directly from Diffusion MRI Tractography](https://mlsanity.com/paper/2505.22685/) (2025)
- [dMRI-F: A Foundation Model for Generalizable Diffusion MRI Analysis from Sparse Acquisitions](https://mlsanity.com/paper/2610.17968/) (2026)

## Abstract (arXiv)

Diffusion-weighted MRI is increasingly used to study the normal and abnormal development of fetal brain in-utero. Recent studies have shown that dMRI can offer invaluable insights into the neurodevelopmental processes in the fetal stage. However, because of the low data quality and rapid brain development, reliable analysis of fetal dMRI data requires dedicated computational methods that are currently unavailable. The lack of automated methods for fast, accurate, and reproducible data analysis has seriously limited our ability to tap the potential of fetal brain dMRI for medical and scientific applications. In this work, we developed and validated a unified computational framework to (1) segment the brain tissue into white matter, cortical/subcortical gray matter, and cerebrospinal fluid, (2) segment 31 distinct white matter tracts, and (3) parcellate the brain's cortex and delineate the deep gray nuclei and white matter structures into 96 anatomically meaningful regions. We utilized a set of manual, semi-automatic, and automatic approaches to annotate 97 fetal brains. Using these labels, we developed and validated a multi-task deep learning method to perform the three computations. Our evaluations show that the new method can accurately carry out all three tasks, achieving a mean Dice similarity coefficient of 0.865 on tissue segmentation, 0.825 on white matter tract segmentation, and 0.819 on parcellation. The proposed method can greatly advance the field of fetal neuroimaging as it can lead to substantial improvements in fetal brain tractography, tract-specific analysis, and structural connectivity assessment.