# Deep Interactive Segmentation of Medical Images: A Systematic Review and Taxonomy

arXiv: https://arxiv.org/abs/2311.13964 · HTML: https://mlsanity.com/paper/2311.13964/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 84 of its 193 references that are in the map.
- First posted on arXiv 23 Nov 2023.

## Builds on (most cited first, 30 of 84)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Segment Anything Model for Medical Image Analysis: an Experimental Study](https://mlsanity.com/paper/2304.10517/) (2023)
- [CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection](https://mlsanity.com/paper/2301.00785/) (2023)
- [Metrics reloaded: Recommendations for image analysis validation](https://mlsanity.com/paper/2206.01653/) (2022)
- [Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging](https://mlsanity.com/paper/2304.04155/) (2023)
- [Deep Interactive Object Selection](https://mlsanity.com/paper/1603.04042/) (2016)
- [Segmentation-Based Deep-Learning Approach for Surface-Defect Detection](https://mlsanity.com/paper/1903.08536/) (2019)
- [Deep Extreme Cut: From Extreme Points to Object Segmentation](https://mlsanity.com/paper/1711.09081/) (2017)
- [f-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation](https://mlsanity.com/paper/2001.10331/) (2020)
- [Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning](https://mlsanity.com/paper/1710.04043/) (2017)
- [DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation](https://mlsanity.com/paper/1707.00652/) (2017)
- [Annotating Object Instances with a Polygon-RNN](https://mlsanity.com/paper/1704.05548/) (2017)
- [SAM on Medical Images: A Comprehensive Study on Three Prompt Modes](https://mlsanity.com/paper/2305.00035/) (2023)
- [DeepCut: Object Segmentation from Bounding Box Annotations using Convolutional Neural Networks](https://mlsanity.com/paper/1605.07866/) (2016)
- [Polyp-SAM: Transfer SAM for Polyp Segmentation](https://mlsanity.com/paper/2305.00293/) (2023)
- [SAM.MD: Zero-shot medical image segmentation capabilities of the Segment Anything Model](https://mlsanity.com/paper/2304.05396/) (2023)
- [Understanding metric-related pitfalls in image analysis validation](https://mlsanity.com/paper/2302.01790/) (2023)
- [Iteratively Trained Interactive Segmentation](https://mlsanity.com/paper/1805.04398/) (2018)
- [When SAM Meets Medical Images: An Investigation of Segment Anything Model (SAM) on Multi-phase Liver Tumor Segmentation](https://mlsanity.com/paper/2304.08506/) (2023)
- [AbdomenAtlas-8K: Annotating 8,000 CT Volumes for Multi-Organ Segmentation in Three Weeks](https://mlsanity.com/paper/2305.09666/) (2023)
- [Generalist Vision Foundation Models for Medical Imaging: A Case Study of Segment Anything Model on Zero-Shot Medical Segmentation](https://mlsanity.com/paper/2304.12637/) (2023)
- [Learning to Segment Medical Images with Scribble-Supervision Alone](https://mlsanity.com/paper/1807.04668/) (2018)

## Built on by (most cited first, 27 of 27)

- [Interactive Medical Image Segmentation: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2411.12814/) (2024)
- [Interactive 3D Medical Image Segmentation with SAM 2](https://mlsanity.com/paper/2408.02635/) (2024)
- [Diffusion-empowered AutoPrompt MedSAM](https://mlsanity.com/paper/2502.06817/) (2025)
- [MedUHIP: Towards Human-In-the-Loop Medical Segmentation](https://mlsanity.com/paper/2408.01620/) (2024)
- [MSDNet: Multi-Scale Decoder for Few-Shot Semantic Segmentation via Transformer-Guided Prototyping](https://mlsanity.com/paper/2409.11316/) (2024)
- [Strike the Balance: On-the-Fly Uncertainty based User Interactions for Long-Term Video Object Segmentation](https://mlsanity.com/paper/2408.00169/) (2024)
- [RadioActive: 3D Radiological Interactive Segmentation Benchmark](https://mlsanity.com/paper/2411.07885/) (2024)
- [Efficient MedSAMs: Segment Anything in Medical Images on Laptop](https://mlsanity.com/paper/2412.16085/) (2024)
- [Is Visual in-Context Learning for Compositional Medical Tasks within Reach?](https://mlsanity.com/paper/2507.00868/) (2025)
- [Rethinking Annotator Simulation: Realistic Evaluation of Whole-Body PET Lesion Interactive Segmentation Methods](https://mlsanity.com/paper/2404.01816/) (2024)
- [IMPACT-CYCLE: A Contract-Based Multi-Agent System for Claim-Level Supervisory Correction of Long-Video Semantic Memory](https://mlsanity.com/paper/2604.20136/) (2026)
- [Whole-Slide Image Analysis under Realistic Few-Shot Annotation Protocols](https://mlsanity.com/paper/2608.30420/) (2026)
- [Interactive 3D Segmentation for Primary Gross Tumor Volume in Oropharyngeal Cancer](https://mlsanity.com/paper/2409.06605/) (2024)
- [Benchmarking Feature Upsampling Methods for Vision Foundation Models using Interactive Segmentation](https://mlsanity.com/paper/2505.02075/) (2025)
- [From Static to Interactive: Adapting Visual in-Context Learners for User-Driven Tasks](https://mlsanity.com/paper/2604.06748/) (2026)
- [EU-Nets: Enhanced, Explainable and Parsimonious U-Nets](https://mlsanity.com/paper/2502.18122/) (2025)
- [RefCut: Interactive Segmentation with Reference Guidance](https://mlsanity.com/paper/2503.17820/) (2025)
- [Weakly Supervised Object Segmentation by Background Conditional Divergence](https://mlsanity.com/paper/2506.22505/) (2025)
- [MOIS-SAM2: Exemplar-based Segment Anything Model 2 for multilesion interactive segmentation of neurofibromas in whole-body MRI](https://mlsanity.com/paper/2509.19277/) (2025)
- [The 1st Solution for CARE Liver Task Challenge 2025: Contrast-Aware Semi-Supervised Segmentation with Domain Generalization and Test-Time Adaptation](https://mlsanity.com/paper/2510.04243/) (2025)
- [VLM-Guided Iterative Refinement for Surgical Image Segmentation with Foundation Models](https://mlsanity.com/paper/2602.09252/) (2026)
- [CataractSAM-2: A Domain-Adapted Model for Anterior Segment Surgery Segmentation and Scalable Ground-Truth Annotation](https://mlsanity.com/paper/2603.21566/) (2026)
- [MedVol-R1: Reward-Driven Evidence Grounding for Volumetric Reasoning Segmentation](https://mlsanity.com/paper/2605.26621/) (2026)
- [Three-Phase Scribble-Adaptive Curriculum Learning for autoPETV Grand Challenge](https://mlsanity.com/paper/2608.22096/) (2026)
- [IMVS: Interactive Medical Volume Segmentation with Test-Time Adaptation - A New Method for Annotating Radiology Datasets](https://mlsanity.com/paper/2609.16775/) (2026)
- [Confidence-Aware Teacher-Student Distillation for 3D Medical Segmentation](https://mlsanity.com/paper/2609.23815/) (2026)
- [From Human Corrections to Learned Segmentations: A Critical Mathematical Review of Information, Learning, and Human Effort](https://mlsanity.com/paper/2610.20396/) (2026)

## Abstract (arXiv)

Interactive segmentation is a crucial research area in medical image analysis aiming to boost the efficiency of costly annotations by incorporating human feedback. This feedback takes the form of clicks, scribbles, or masks and allows for iterative refinement of the model output so as to efficiently guide the system towards the desired behavior. In recent years, deep learning-based approaches have propelled results to a new level causing a rapid growth in the field with 121 methods proposed in the medical imaging domain alone. In this review, we provide a structured overview of this emerging field featuring a comprehensive taxonomy, a systematic review of existing methods, and an in-depth analysis of current practices. Based on these contributions, we discuss the challenges and opportunities in the field. For instance, we find that there is a severe lack of comparison across methods which needs to be tackled by standardized baselines and benchmarks.