# DeepEdit: Deep Editable Learning for Interactive Segmentation of 3D Medical Images

arXiv: https://arxiv.org/abs/2305.10655 · HTML: https://mlsanity.com/paper/2305.10655/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [Deep Interactive Object Selection](https://mlsanity.com/paper/1603.04042/) (2016)
- [Deep Extreme Cut: From Extreme Points to Object Segmentation](https://mlsanity.com/paper/1711.09081/) (2017)
- [Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning](https://mlsanity.com/paper/1710.04043/) (2017)
- [DiNTS: Differentiable Neural Network Topology Search for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.15954/) (2021)
- [Interactive Full Image Segmentation by Considering All Regions Jointly](https://mlsanity.com/paper/1812.01888/) (2018)
- [Interactive segmentation of medical images through fully convolutional neural networks](https://mlsanity.com/paper/1903.08205/) (2019)

## Built on by (most cited first, 13 of 13)

- [Deep Interactive Segmentation of Medical Images: A Systematic Review and Taxonomy](https://mlsanity.com/paper/2311.13964/) (2023)
- [mathrmSAMMed: A medical image annotation framework based on large vision model](https://mlsanity.com/paper/2307.05617/) (2023)
- [Sliding Window FastEdit: A Framework for Lesion Annotation in Whole-body PET Images](https://mlsanity.com/paper/2311.14482/) (2023)
- [Conformal Risk Control for Pulmonary Nodule Detection](https://mlsanity.com/paper/2412.20167/) (2024)
- [SAM3D: Zero-Shot Semi-Automatic Segmentation in 3D Medical Images with the Segment Anything Model](https://mlsanity.com/paper/2405.06786/) (2024)
- [Improving Uncertainty-Error Correspondence in Deep Bayesian Medical Image Segmentation](https://mlsanity.com/paper/2409.03470/) (2024)
- [Interactive 3D Segmentation for Primary Gross Tumor Volume in Oropharyngeal Cancer](https://mlsanity.com/paper/2409.06605/) (2024)
- [Anatomy-Informed Deep Learning and Radiomics for Automated Neurofibroma Segmentation in Whole-Body MRI](https://mlsanity.com/paper/2502.15424/) (2025)
- [Implicit Shape-Prior for Few-Shot Assisted 3D Segmentation](https://mlsanity.com/paper/2509.08580/) (2025)
- [MOIS-SAM2: Exemplar-based Segment Anything Model 2 for multilesion interactive segmentation of neurofibromas in whole-body MRI](https://mlsanity.com/paper/2509.19277/) (2025)
- [CytoCrowd: A Multi-Annotator Benchmark Dataset for Cytology Image Analysis](https://mlsanity.com/paper/2602.06674/) (2026)
- [Interactive Medical-SAM2 GUI: A Napari-based semi-automatic annotation tool for medical images](https://mlsanity.com/paper/2602.22649/) (2026)
- [A Clinician-Centered Pipeline for Annotation and Evaluation in Ultrasound AI Studies](https://mlsanity.com/paper/2606.19174/) (2026)

## Abstract (arXiv)

Automatic segmentation of medical images is a key step for diagnostic and interventional tasks. However, achieving this requires large amounts of annotated volumes, which can be tedious and time-consuming task for expert annotators. In this paper, we introduce DeepEdit, a deep learning-based method for volumetric medical image annotation, that allows automatic and semi-automatic segmentation, and click-based refinement. DeepEdit combines the power of two methods: a non-interactive (i.e. automatic segmentation using nnU-Net, UNET or UNETR) and an interactive segmentation method (i.e. DeepGrow), into a single deep learning model. It allows easy integration of uncertainty-based ranking strategies (i.e. aleatoric and epistemic uncertainty computation) and active learning. We propose and implement a method for training DeepEdit by using standard training combined with user interaction simulation. Once trained, DeepEdit allows clinicians to quickly segment their datasets by using the algorithm in auto segmentation mode or by providing clicks via a user interface (i.e. 3D Slicer, OHIF). We show the value of DeepEdit through evaluation on the PROSTATEx dataset for prostate/prostatic lesions and the Multi-Atlas Labeling Beyond the Cranial Vault (BTCV) dataset for abdominal CT segmentation, using state-of-the-art network architectures as baseline for comparison. DeepEdit could reduce the time and effort annotating 3D medical images compared to DeepGrow alone. Source code is available at https://github.com/Project-MONAI/MONAILabel