# DeepNAT: Deep Convolutional Neural Network for Segmenting Neuroanatomy

arXiv: https://arxiv.org/abs/1702.08192 · HTML: https://mlsanity.com/paper/1702.08192/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2017.

## Built on by (most cited first, 25 of 25)

- [End-to-end Prostate Cancer Detection in bpMRI via 3D CNNs: Effects of Attention Mechanisms, Clinical Priori and Decoupled False Positive Reduction](https://mlsanity.com/paper/2101.03244/) (2021)
- [Neuralizer: General Neuroimage Analysis without Re-Training](https://mlsanity.com/paper/2305.02644/) (2023)
- [FastSurferVINN: Building Resolution-Independence into Deep Learning Segmentation Methods -- A Solution for HighRes Brain MRI](https://mlsanity.com/paper/2112.09654/) (2021)
- [A survey on shape-constraint deep learning for medical image segmentation](https://mlsanity.com/paper/2101.07721/) (2021)
- [Challenges for machine learning in clinical translation of big data imaging studies](https://mlsanity.com/paper/2107.05630/) (2021)
- [Neural deformation fields for template-based reconstruction of cortical surfaces from MRI](https://mlsanity.com/paper/2401.12938/) (2024)
- [FAST-AID Brain: Fast and Accurate Segmentation Tool using Artificial Intelligence Developed for Brain](https://mlsanity.com/paper/2208.14360/) (2022)
- [Whole Brain Segmentation with Full Volume Neural Network](https://mlsanity.com/paper/2110.15601/) (2021)
- [Fighting the scanner effect in brain MRI segmentation with a progressive level-of-detail network trained on multi-site data](https://mlsanity.com/paper/2211.02400/) (2022)
- [Longitudinal Quantitative Assessment of COVID-19 Infection Progression from Chest CTs](https://mlsanity.com/paper/2103.07240/) (2021)
- [Automated computed tomography and magnetic resonance imaging segmentation using deep learning: a beginner's guide](https://mlsanity.com/paper/2304.05901/) (2023)
- [Suggestive Annotation of Brain MR Images with Gradient-guided Sampling](https://mlsanity.com/paper/2206.01014/) (2022)
- [Label merge-and-split: A graph-colouring approach for memory-efficient brain parcellation](https://mlsanity.com/paper/2404.10572/) (2024)
- [Explaining the Success of Nearest Neighbor Methods in Prediction](https://mlsanity.com/paper/2502.15900/) (2025)
- [SIAM: Head and Brain MRI Segmentation from Few High-Quality Templates via Synthetic Training](https://mlsanity.com/paper/2605.02737/) (2026)
- [Why Calibration Error is Wrong Given Model Uncertainty: Using Posterior Predictive Checks with Deep Learning](https://mlsanity.com/paper/2112.01477/) (2021)
- [VINNA for Neonates -- Orientation Independence through Latent Augmentations](https://mlsanity.com/paper/2311.17546/) (2023)
- [Three-dimensional end-to-end deep learning for brain MRI analysis](https://mlsanity.com/paper/2506.23916/) (2025)
- [HEAL: Learning-Free Source Free Unsupervised Domain Adaptation for Cross-Modality Medical Image Segmentation](https://mlsanity.com/paper/2511.17958/) (2025)
- [Factorisation-based Image Labelling](https://mlsanity.com/paper/2111.10326/) (2021)
- [Domain-agnostic segmentation of thalamic nuclei from joint structural and diffusion MRI](https://mlsanity.com/paper/2305.03413/) (2023)
- [Enhancing Hierarchical Transformers for Whole Brain Segmentation with Intracranial Measurements Integration](https://mlsanity.com/paper/2309.04071/) (2023)
- [DeepThalamus: A novel deep learning method for automatic segmentation of brain thalamic nuclei from multimodal ultra-high resolution MRI](https://mlsanity.com/paper/2401.07751/) (2024)
- [AI for the prediction of early stages of Alzheimer's disease from neuroimaging biomarkers -- A narrative review of a growing field](https://mlsanity.com/paper/2406.17822/) (2024)
- [Beyond one-hot encoding? Journey into compact encoding for large multi-class segmentation](https://mlsanity.com/paper/2510.00667/) (2025)

## Abstract (arXiv)

We introduce DeepNAT, a 3D Deep convolutional neural network for the automatic segmentation of NeuroAnaTomy in T1-weighted magnetic resonance images. DeepNAT is an end-to-end learning-based approach to brain segmentation that jointly learns an abstract feature representation and a multi-class classification. We propose a 3D patch-based approach, where we do not only predict the center voxel of the patch but also neighbors, which is formulated as multi-task learning. To address a class imbalance problem, we arrange two networks hierarchically, where the first one separates foreground from background, and the second one identifies 25 brain structures on the foreground. Since patches lack spatial context, we augment them with coordinates. To this end, we introduce a novel intrinsic parameterization of the brain volume, formed by eigenfunctions of the Laplace-Beltrami operator. As network architecture, we use three convolutional layers with pooling, batch normalization, and non-linearities, followed by fully connected layers with dropout. The final segmentation is inferred from the probabilistic output of the network with a 3D fully connected conditional random field, which ensures label agreement between close voxels. The roughly 2.7 million parameters in the network are learned with stochastic gradient descent. Our results show that DeepNAT compares favorably to state-of-the-art methods. Finally, the purely learning-based method may have a high potential for the adaptation to young, old, or diseased brains by fine-tuning the pre-trained network with a small training sample on the target application, where the availability of larger datasets with manual annotations may boost the overall segmentation accuracy in the future.