# One Network to Segment Them All: A General, Lightweight System for Accurate 3D Medical Image Segmentation

arXiv: https://arxiv.org/abs/1911.01764 · HTML: https://mlsanity.com/paper/1911.01764/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2019.

## Built on by (most cited first, 25 of 25)

- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [DiNTS: Differentiable Neural Network Topology Search for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.15954/) (2021)
- [Scheduling Techniques for Liver Segmentation: ReduceLRonPlateau Vs OneCycleLR](https://mlsanity.com/paper/2202.06373/) (2022)
- [Towards Robust General Medical Image Segmentation](https://mlsanity.com/paper/2107.04263/) (2021)
- [T-AutoML: Automated Machine Learning for Lesion Segmentation using Transformers in 3D Medical Imaging](https://mlsanity.com/paper/2111.07535/) (2021)
- [FAST-AID Brain: Fast and Accurate Segmentation Tool using Artificial Intelligence Developed for Brain](https://mlsanity.com/paper/2208.14360/) (2022)
- [OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data](https://mlsanity.com/paper/2110.10640/) (2021)
- [SegICL: A Multimodal In-context Learning Framework for Enhanced Segmentation in Medical Imaging](https://mlsanity.com/paper/2403.16578/) (2024)
- [Predicting Knee Osteoarthritis Progression from Structural MRI using Deep Learning](https://mlsanity.com/paper/2201.10849/) (2022)
- [Automated liver tissues delineation techniques: A systematic survey on machine learning current trends and future orientations](https://mlsanity.com/paper/2103.06384/) (2021)
- [OCTAMamba: A State-Space Model Approach for Precision OCTA Vasculature Segmentation](https://mlsanity.com/paper/2409.08000/) (2024)
- [Patch-based Medical Image Segmentation using Matrix Product State Tensor Networks](https://mlsanity.com/paper/2109.07138/) (2021)
- [Exploring Generalizable Distillation for Efficient Medical Image Segmentation](https://mlsanity.com/paper/2207.12995/) (2022)
- [Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation](https://mlsanity.com/paper/2302.03296/) (2023)
- [Quantifying Impairment and Disease Severity Using AI Models Trained on Healthy Subjects](https://mlsanity.com/paper/2311.12781/) (2023)
- [AutoSeg -- Steering the Inductive Biases for Automatic Pathology Segmentation](https://mlsanity.com/paper/2201.09579/) (2022)
- [UNet Architectures in Multiplanar Volumetric Segmentation -- Validated on Three Knee MRI Cohorts](https://mlsanity.com/paper/2203.08194/) (2022)
- [Discover the Mysteries of the Maya: Selected Contributions from the Machine Learning Challenge & The Discovery Challenge Workshop at ECML PKDD 2021](https://mlsanity.com/paper/2208.03163/) (2022)
- [Crosslink-Net: Double-branch Encoder Segmentation Network via Fusing Vertical and Horizontal Convolutions](https://mlsanity.com/paper/2107.11517/) (2021)
- [Auto-segmentation of Hip Joints using MultiPlanar UNet with Transfer learning](https://mlsanity.com/paper/2208.08226/) (2022)
- [Fully Automated Tumor Segmentation for Brain MRI data using Multiplanner UNet](https://mlsanity.com/paper/2401.06499/) (2024)
- [FALCON: Few-Shot Adversarial Learning for Cross-Domain Medical Image Segmentation](https://mlsanity.com/paper/2601.01687/) (2026)
- [Optimization in Sparse 2D to Dense 3D Weakly Supervised Learning: Application to Multi-Label Segmentation of Large ex vivo MRI Data](https://mlsanity.com/paper/2605.12753/) (2026)
- [Parameter-Efficient 3D Segmentation of Liver and Liver tumors: Depthwise factorization Scales Better Than Dense Convolution with Spatial Dimensionality](https://mlsanity.com/paper/2609.33077/) (2026)
- [Active-DiNTS: Active Differentiable Network Topology Search](https://mlsanity.com/paper/2610.04787/) (2026)

## Abstract (arXiv)

Many recent medical segmentation systems rely on powerful deep learning models to solve highly specific tasks. To maximize performance, it is standard practice to evaluate numerous pipelines with varying model topologies, optimization parameters, pre- & postprocessing steps, and even model cascades. It is often not clear how the resulting pipeline transfers to different tasks. We propose a simple and thoroughly evaluated deep learning framework for segmentation of arbitrary medical image volumes. The system requires no task-specific information, no human interaction and is based on a fixed model topology and a fixed hyperparameter set, eliminating the process of model selection and its inherent tendency to cause method-level over-fitting. The system is available in open source and does not require deep learning expertise to use. Without task-specific modifications, the system performed better than or similar to highly specialized deep learning methods across 3 separate segmentation tasks. In addition, it ranked 5-th and 6-th in the first and second round of the 2018 Medical Segmentation Decathlon comprising another 10 tasks. The system relies on multi-planar data augmentation which facilitates the application of a single 2D architecture based on the familiar U-Net. Multi-planar training combines the parameter efficiency of a 2D fully convolutional neural network with a systematic train- and test-time augmentation scheme, which allows the 2D model to learn a representation of the 3D image volume that fosters generalization.