# Catalyzing Clinical Diagnostic Pipelines Through Volumetric Medical Image Segmentation Using Deep Neural Networks: Past, Present, & Future

arXiv: https://arxiv.org/abs/2103.14969 · HTML: https://mlsanity.com/paper/2103.14969/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 61 references that are in the map.
- First posted on arXiv 27 Mar 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [3D MRI brain tumor segmentation using autoencoder regularization](https://mlsanity.com/paper/1810.11654/) (2018)
- [The Importance of Skip Connections in Biomedical Image Segmentation](https://mlsanity.com/paper/1608.04117/) (2016)
- [No New-Net](https://mlsanity.com/paper/1809.10483/) (2018)
- [Ensembles of Multiple Models and Architectures for Robust Brain Tumour Segmentation](https://mlsanity.com/paper/1711.01468/) (2017)
- [3D Convolutional Encoder-Decoder Network for Low-Dose CT via Transfer Learning from a 2D Trained Network](https://mlsanity.com/paper/1802.05656/) (2018)
- [Hough-CNN: Deep Learning for Segmentation of Deep Brain Regions in MRI and Ultrasound](https://mlsanity.com/paper/1601.07014/) (2016)
- [Fine-grained Recurrent Neural Networks for Automatic Prostate Segmentation in Ultrasound Images](https://mlsanity.com/paper/1612.01655/) (2016)
- [VoxelAtlasGAN: 3D Left Ventricle Segmentation on Echocardiography with Atlas Guided Generation and Voxel-to-voxel Discrimination](https://mlsanity.com/paper/1806.03619/) (2018)
- [Deep learning with spatiotemporal consistency for nerve segmentation in ultrasound images](https://mlsanity.com/paper/1706.05870/) (2017)
- [Multi-Scale Convolutional-Stack Aggregation for Robust White Matter Hyperintensities Segmentation](https://mlsanity.com/paper/1807.05153/) (2018)
- [SUMNet: Fully Convolutional Model for Fast Segmentation of Anatomical Structures in Ultrasound Volumes](https://mlsanity.com/paper/1901.06920/) (2019)

## Abstract (arXiv)

Deep learning has made a remarkable impact in the field of natural image processing over the past decade. Consequently, there is a great deal of interest in replicating this success across unsolved tasks in related domains, such as medical image analysis. Core to medical image analysis is the task of semantic segmentation which enables various clinical workflows. Due to the challenges inherent in manual segmentation, many decades of research have been devoted to discovering extensible, automated, expert-level segmentation techniques. Given the groundbreaking performance demonstrated by recent neural network-based techniques, deep learning seems poised to achieve what classic methods have historically been unable. This paper will briefly overview some of the state-of-the-art (SoTA) neural network-based segmentation algorithms with a particular emphasis on the most recent architectures, comparing and contrasting the contributions and characteristics of each network topology. Using ultrasonography as a motivating example, it will also demonstrate important clinical implications of effective deep learning-based solutions, articulate challenges unique to the modality, and discuss novel approaches developed in response to those challenges, concluding with the proposal of future directions in the field. Given the generally observed ephemerality of the best deep learning approaches (i.e. the extremely quick succession of the SoTA), the main contributions of the paper are its contextualization of modern deep learning architectures with historical background and the elucidation of the current trajectory of volumetric medical image segmentation research.