# ScribblePrompt: Fast and Flexible Interactive Segmentation for Any Biomedical Image

arXiv: https://arxiv.org/abs/2312.07381 · HTML: https://mlsanity.com/paper/2312.07381/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 70 of its 145 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Builds on (most cited first, 30 of 70)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1710.05006/) (2017)
- [Segment Everything Everywhere All at Once](https://mlsanity.com/paper/2304.06718/) (2023)
- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [Transfusion: Understanding Transfer Learning for Medical Imaging](https://mlsanity.com/paper/1902.07208/) (2019)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [Images Speak in Images: A Generalist Painter for In-Context Visual Learning](https://mlsanity.com/paper/2212.02499/) (2022)
- [CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation](https://mlsanity.com/paper/2001.06535/) (2020)
- [Segment Anything Model for Medical Image Analysis: an Experimental Study](https://mlsanity.com/paper/2304.10517/) (2023)
- [Customized Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.13785/) (2023)
- [Deep Learning for Segmentation using an Open Large-Scale Dataset in 2D Echocardiography](https://mlsanity.com/paper/1908.06948/) (2019)
- [Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge](https://mlsanity.com/paper/1612.08012/) (2016)
- [The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challenge](https://mlsanity.com/paper/1912.01054/) (2019)
- [Gradio: Hassle-Free Sharing and Testing of ML Models in the Wild](https://mlsanity.com/paper/1906.02569/) (2019)
- [Reviving Iterative Training with Mask Guidance for Interactive Segmentation](https://mlsanity.com/paper/2102.06583/) (2021)
- [Deep Interactive Object Selection](https://mlsanity.com/paper/1603.04042/) (2016)
- [Deep Extreme Cut: From Extreme Points to Object Segmentation](https://mlsanity.com/paper/1711.09081/) (2017)

## Built on by (most cited first, 30 of 48)

- [nnInteractive: Redefining 3D Promptable Segmentation](https://mlsanity.com/paper/2503.08373/) (2025)
- [MedCLIP-SAMv2: Towards Universal Text-Driven Medical Image Segmentation](https://mlsanity.com/paper/2409.19483/) (2024)
- [LesionLocator: Zero-Shot Universal Tumor Segmentation and Tracking in 3D Whole-Body Imaging](https://mlsanity.com/paper/2502.20985/) (2025)
- [SegAnyPET: Universal Promptable Segmentation from Positron Emission Tomography Images](https://mlsanity.com/paper/2502.14351/) (2025)
- [MediSee: Reasoning-based Pixel-level Perception in Medical Images](https://mlsanity.com/paper/2504.11008/) (2025)
- [LiVOS: Light Video Object Segmentation with Gated Linear Matching](https://mlsanity.com/paper/2411.02818/) (2024)
- [Sli2Vol+: Segmenting 3D Medical Images Based on an Object Estimation Guided Correspondence Flow Network](https://mlsanity.com/paper/2411.13873/) (2024)
- [RadioActive: 3D Radiological Interactive Segmentation Benchmark](https://mlsanity.com/paper/2411.07885/) (2024)
- [MCP-MedSAM: A Powerful Lightweight Medical Segment Anything Model Trained with a Single GPU in Just One Day](https://mlsanity.com/paper/2412.05888/) (2024)
- [Boltzmann Attention Sampling for Image Analysis with Small Objects](https://mlsanity.com/paper/2503.02841/) (2025)
- [MedCLIPSeg: Probabilistic Vision-Language Adaptation for Data-Efficient and Generalizable Medical Image Segmentation](https://mlsanity.com/paper/2602.20423/) (2026)
- [Segment Anything Model for automated image data annotation: empirical studies using text prompts from Grounding DINO](https://mlsanity.com/paper/2406.19057/) (2024)
- [Medverse: A Universal Model for Full-Resolution 3D Medical Image Segmentation, Transformation and Enhancement](https://mlsanity.com/paper/2509.09232/) (2025)
- [Atlas is Your Perfect Context: One-Shot Customization for Generalizable Foundational Medical Image Segmentation](https://mlsanity.com/paper/2512.18176/) (2025)
- [MedicalNarratives: Connecting Medical Vision and Language with Localized Narratives](https://mlsanity.com/paper/2501.04184/) (2025)
- [RP-SAM2: Refining Point Prompts for Stable Surgical Instrument Segmentation](https://mlsanity.com/paper/2504.07117/) (2025)
- [Promptable cancer segmentation using minimal expert-curated data](https://mlsanity.com/paper/2505.17915/) (2025)
- [PolyPose: Deformable 2D/3D Registration via Polyrigid Transformations](https://mlsanity.com/paper/2505.19256/) (2025)
- [Revisiting 3D Medical Scribble Supervision: Benchmarking Beyond Cardiac Segmentation](https://mlsanity.com/paper/2403.12834/) (2024)
- [LIMIS: Towards Language-based Interactive Medical Image Segmentation](https://mlsanity.com/paper/2410.16939/) (2024)
- [Generative AI in Medicine](https://mlsanity.com/paper/2412.10337/) (2024)
- [SketchYourSeg: Mask-Free Subjective Image Segmentation via Freehand Sketches](https://mlsanity.com/paper/2501.16022/) (2025)
- [AortaDiff: Volume-Guided Conditional Diffusion Models for Multi-Branch Aortic Surface Generation](https://mlsanity.com/paper/2507.13404/) (2025)
- [Medal S: Spatio-Textual Prompt Model for Medical Segmentation](https://mlsanity.com/paper/2511.13001/) (2025)
- [SurgSLOT: Segment Anything in Surgical Videos via Semantic Long-term Tracking](https://mlsanity.com/paper/2511.16618/) (2025)
- [Expert-like Bone Ultrasound Segmentation through Expert-in-the-loop Mask-conditioned Progressive Learning](https://mlsanity.com/paper/2609.00473/) (2026)
- [Improved Baselines with Synchronized Encoding for Universal Medical Image Segmentation](https://mlsanity.com/paper/2408.09886/) (2024)
- [Reservoir-enhanced Segment Anything Model for Subsurface Diagnosis](https://mlsanity.com/paper/2504.18802/) (2025)
- [Brain Imaging Foundation Models, Are We There Yet? A Systematic Review of Foundation Models for Brain Imaging and Biomedical Research](https://mlsanity.com/paper/2506.13306/) (2025)
- [Automated Segmentation of Coronal Brain Tissue Slabs for 3D Neuropathology](https://mlsanity.com/paper/2508.09805/) (2025)

## Abstract (arXiv)

Biomedical image segmentation is a crucial part of both scientific research and clinical care. With enough labelled data, deep learning models can be trained to accurately automate specific biomedical image segmentation tasks. However, manually segmenting images to create training data is highly labor intensive and requires domain expertise. We present ScribblePrompt, a flexible neural network based interactive segmentation tool for biomedical imaging that enables human annotators to segment previously unseen structures using scribbles, clicks, and bounding boxes. Through rigorous quantitative experiments, we demonstrate that given comparable amounts of interaction, ScribblePrompt produces more accurate segmentations than previous methods on datasets unseen during training. In a user study with domain experts, ScribblePrompt reduced annotation time by 28% while improving Dice by 15% compared to the next best method. ScribblePrompt's success rests on a set of careful design decisions. These include a training strategy that incorporates both a highly diverse set of images and tasks, novel algorithms for simulated user interactions and labels, and a network that enables fast inference. We showcase ScribblePrompt in an interactive demo, provide code, and release a dataset of scribble annotations at https://scribbleprompt.csail.mit.edu