# AnatomyNet: Deep Learning for Fast and Fully Automated Whole-volume Segmentation of Head and Neck Anatomy

arXiv: https://arxiv.org/abs/1808.05238 · HTML: https://mlsanity.com/paper/1808.05238/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Aug 2018.

## Built on by (most cited first, 30 of 54)

- [Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentation](https://mlsanity.com/paper/2102.04525/) (2021)
- [Modality-aware Mutual Learning for Multi-modal Medical Image Segmentation](https://mlsanity.com/paper/2107.09842/) (2021)
- [SegRap2023: A Benchmark of Organs-at-Risk and Gross Tumor Volume Segmentation for Radiotherapy Planning of Nasopharyngeal Carcinoma](https://mlsanity.com/paper/2312.09576/) (2023)
- [YOLOPv2: Better, Faster, Stronger for Panoptic Driving Perception](https://mlsanity.com/paper/2208.11434/) (2022)
- [Segment Anything Model (SAM) for Radiation Oncology](https://mlsanity.com/paper/2306.11730/) (2023)
- [AutoShot: A Short Video Dataset and State-of-the-Art Shot Boundary Detection](https://mlsanity.com/paper/2304.06116/) (2023)
- [blob loss: instance imbalance aware loss functions for semantic segmentation](https://mlsanity.com/paper/2205.08209/) (2022)
- [Test-Time Training for Deformable Multi-Scale Image Registration](https://mlsanity.com/paper/2103.13578/) (2021)
- [FocusNetv2: Imbalanced Large and Small Organ Segmentation with Adversarial Shape Constraint for Head and Neck CT Images](https://mlsanity.com/paper/2104.01771/) (2021)
- [Human Treelike Tubular Structure Segmentation: A Comprehensive Review and Future Perspectives](https://mlsanity.com/paper/2207.11203/) (2022)
- [A Generalized Surface Loss for Reducing the Hausdorff Distance in Medical Imaging Segmentation](https://mlsanity.com/paper/2302.03868/) (2023)
- [FDA: Feature Decomposition and Aggregation for Robust Airway Segmentation](https://mlsanity.com/paper/2109.02920/) (2021)
- [Automatic Segmentation of Organs-at-Risk from Head-and-Neck CT using Separable Convolutional Neural Network with Hard-Region-Weighted Loss](https://mlsanity.com/paper/2102.01897/) (2021)
- [Multi-site, Multi-domain Airway Tree Modeling (ATM'22): A Public Benchmark for Pulmonary Airway Segmentation](https://mlsanity.com/paper/2303.05745/) (2023)
- [A Novel Hybrid Convolutional Neural Network for Accurate Organ Segmentation in 3D Head and Neck CT Images](https://mlsanity.com/paper/2109.12634/) (2021)
- [Tackling the Class Imbalance Problem of Deep Learning Based Head and Neck Organ Segmentation](https://mlsanity.com/paper/2201.01636/) (2022)
- [Automatic Segmentation of Gross Target Volume of Nasopharynx Cancer using Ensemble of Multiscale Deep Neural Networks with Spatial Attention](https://mlsanity.com/paper/2101.11254/) (2021)
- [Tempera: Spatial Transformer Feature Pyramid Network for Cardiac MRI Segmentation](https://mlsanity.com/paper/2203.00355/) (2022)
- [MedUHIP: Towards Human-In-the-Loop Medical Segmentation](https://mlsanity.com/paper/2408.01620/) (2024)
- [Weaving Attention U-net: A Novel Hybrid CNN and Attention-based Method for Organs-at-risk Segmentation in Head and Neck CT Images](https://mlsanity.com/paper/2107.04847/) (2021)
- [Recurrence-free Survival Prediction under the Guidance of Automatic Gross Tumor Volume Segmentation for Head and Neck Cancers](https://mlsanity.com/paper/2209.11268/) (2022)
- [Towards more precise automatic analysis: a comprehensive survey of deep learning-based multi-organ segmentation](https://mlsanity.com/paper/2303.00232/) (2023)
- [Meta-Analysis of Transfer Learning for Segmentation of Brain Lesions](https://mlsanity.com/paper/2306.11714/) (2023)
- [Segmentation and Risk Score Prediction of Head and Neck Cancers in PET/CT Volumes with 3D U-Net and Cox Proportional Hazard Neural Networks](https://mlsanity.com/paper/2202.07823/) (2022)
- [Parotid Gland MRI Segmentation Based on Swin-Unet and Multimodal Images](https://mlsanity.com/paper/2206.03336/) (2022)
- [FAS-UNet: A Novel FAS-driven Unet to Learn Variational Image Segmentation](https://mlsanity.com/paper/2210.15164/) (2022)
- [Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation](https://mlsanity.com/paper/2302.03296/) (2023)
- [EndoDepthL: Lightweight Endoscopic Monocular Depth Estimation with CNN-Transformer](https://mlsanity.com/paper/2308.02716/) (2023)
- [Fully Automatic Segmentation of Gross Target Volume and Organs-at-Risk for Radiotherapy Planning of Nasopharyngeal Carcinoma](https://mlsanity.com/paper/2310.02972/) (2023)
- [PSTNet: Enhanced Polyp Segmentation with Multi-scale Alignment and Frequency Domain Integration](https://mlsanity.com/paper/2409.08501/) (2024)

## Abstract (arXiv)

Methods: Our deep learning model, called AnatomyNet, segments OARs from head and neck CT images in an end-to-end fashion, receiving whole-volume HaN CT images as input and generating masks of all OARs of interest in one shot. AnatomyNet is built upon the popular 3D U-net architecture, but extends it in three important ways: 1) a new encoding scheme to allow auto-segmentation on whole-volume CT images instead of local patches or subsets of slices, 2) incorporating 3D squeeze-and-excitation residual blocks in encoding layers for better feature representation, and 3) a new loss function combining Dice scores and focal loss to facilitate the training of the neural model. These features are designed to address two main challenges in deep-learning-based HaN segmentation: a) segmenting small anatomies (i.e., optic chiasm and optic nerves) occupying only a few slices, and b) training with inconsistent data annotations with missing ground truth for some anatomical structures. Results: We collected 261 HaN CT images to train AnatomyNet, and used MICCAI Head and Neck Auto Segmentation Challenge 2015 as a benchmark dataset to evaluate the performance of AnatomyNet. The objective is to segment nine anatomies: brain stem, chiasm, mandible, optic nerve left, optic nerve right, parotid gland left, parotid gland right, submandibular gland left, and submandibular gland right. Compared to previous state-of-the-art results from the MICCAI 2015 competition, AnatomyNet increases Dice similarity coefficient by 3.3% on average. AnatomyNet takes about 0.12 seconds to fully segment a head and neck CT image of dimension 178 x 302 x 225, significantly faster than previous methods. In addition, the model is able to process whole-volume CT images and delineate all OARs in one pass, requiring little pre- or post-processing. https://github.com/wentaozhu/AnatomyNet-for-anatomical-segmentation.git.