# ScribbleSeg: Scribble-based Interactive Image Segmentation

arXiv: https://arxiv.org/abs/2303.11320 · HTML: https://mlsanity.com/paper/2303.11320/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 20 Mar 2023.

## Builds on (most cited first, 18 of 18)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1604.05144/) (2016)
- [Reviving Iterative Training with Mask Guidance for Interactive Segmentation](https://mlsanity.com/paper/2102.06583/) (2021)
- [Deep Interactive Object Selection](https://mlsanity.com/paper/1603.04042/) (2016)
- [Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion](https://mlsanity.com/paper/2103.07941/) (2021)
- [f-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation](https://mlsanity.com/paper/2001.10331/) (2020)
- [DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation](https://mlsanity.com/paper/1707.00652/) (2017)
- [FocalClick: Towards Practical Interactive Image Segmentation](https://mlsanity.com/paper/2204.02574/) (2022)
- [Iteratively Trained Interactive Segmentation](https://mlsanity.com/paper/1805.04398/) (2018)
- [The 2018 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1803.00557/) (2018)
- [PseudoClick: Interactive Image Segmentation with Click Imitation](https://mlsanity.com/paper/2207.05282/) (2022)
- [Interactive Full Image Segmentation by Considering All Regions Jointly](https://mlsanity.com/paper/1812.01888/) (2018)
- [Fast User-Guided Video Object Segmentation by Interaction-and-Propagation Networks](https://mlsanity.com/paper/1904.09791/) (2019)
- [Interactive Video Object Segmentation Using Global and Local Transfer Modules](https://mlsanity.com/paper/2007.08139/) (2020)
- [Guided Interactive Video Object Segmentation Using Reliability-Based Attention Maps](https://mlsanity.com/paper/2104.10386/) (2021)
- [Efficient Full Image Interactive Segmentation by Leveraging Within-image Appearance Similarity](https://mlsanity.com/paper/2007.08173/) (2020)

## Built on by (most cited first, 10 of 10)

- [ScribblePrompt: Fast and Flexible Interactive Segmentation for Any Biomedical Image](https://mlsanity.com/paper/2312.07381/) (2023)
- [PRISM: A Promptable and Robust Interactive Segmentation Model with Visual Prompts](https://mlsanity.com/paper/2404.15028/) (2024)
- [AdaptiveClick: Clicks-aware Transformer with Adaptive Focal Loss for Interactive Image Segmentation](https://mlsanity.com/paper/2305.04276/) (2023)
- [Adapting the Segment Anything Model During Usage in Novel Situations](https://mlsanity.com/paper/2404.08421/) (2024)
- [Upsampling DINOv2 features for unsupervised vision tasks and weakly supervised materials segmentation](https://mlsanity.com/paper/2410.19836/) (2024)
- [Referring Human Pose and Mask Estimation in the Wild](https://mlsanity.com/paper/2410.20508/) (2024)
- [PVPUFormer: Probabilistic Visual Prompt Unified Transformer for Interactive Image Segmentation](https://mlsanity.com/paper/2306.06656/) (2023)
- [Interactive Occlusion Boundary Estimation through Exploitation of Synthetic Data](https://mlsanity.com/paper/2408.15038/) (2024)
- [Scribble-Guided Diffusion for Training-free Text-to-Image Generation](https://mlsanity.com/paper/2409.08026/) (2024)
- [RoadGIE: Towards A Global-Scale Aerial Benchmark for Generalizable Interactive Road Extraction](https://mlsanity.com/paper/2605.26862/) (2026)

## Abstract (arXiv)

Interactive segmentation enables users to extract masks by providing simple annotations to indicate the target, such as boxes, clicks, or scribbles. Among these interaction formats, scribbles are the most flexible as they can be of arbitrary shapes and sizes. This enables scribbles to provide more indications of the target object. However, previous works mainly focus on click-based configuration, and the scribble-based setting is rarely explored. In this work, we attempt to formulate a standard protocol for scribble-based interactive segmentation. Basically, we design diversified strategies to simulate scribbles for training, propose a deterministic scribble generator for evaluation, and construct a challenging benchmark. Besides, we build a strong framework ScribbleSeg, consisting of a Prototype Adaption Module(PAM) and a Corrective Refine Module (CRM), for the task. Extensive experiments show that ScribbleSeg performs notably better than previous click-based methods. We hope this could serve as a more powerful and general solution for interactive segmentation. Our code will be made available.