# Interactive Full Image Segmentation by Considering All Regions Jointly

arXiv: https://arxiv.org/abs/1812.01888 · HTML: https://mlsanity.com/paper/1812.01888/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2018.

## Built on by (most cited first, 30 of 34)

- [InstanceDiffusion: Instance-level Control for Image Generation](https://mlsanity.com/paper/2402.03290/) (2024)
- [Reviving Iterative Training with Mask Guidance for Interactive Segmentation](https://mlsanity.com/paper/2102.06583/) (2021)
- [Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion](https://mlsanity.com/paper/2103.07941/) (2021)
- [ScribblePrompt: Fast and Flexible Interactive Segmentation for Any Biomedical Image](https://mlsanity.com/paper/2312.07381/) (2023)
- [MONAI Label: A framework for AI-assisted Interactive Labeling of 3D Medical Images](https://mlsanity.com/paper/2203.12362/) (2022)
- [AGILE3D: Attention Guided Interactive Multi-object 3D Segmentation](https://mlsanity.com/paper/2306.00977/) (2023)
- [InterFormer: Real-time Interactive Image Segmentation](https://mlsanity.com/paper/2304.02942/) (2023)
- [DIAL: Deep Interactive and Active Learning for Semantic Segmentation in Remote Sensing](https://mlsanity.com/paper/2201.01047/) (2022)
- [Interactive Multi-Class Tiny-Object Detection](https://mlsanity.com/paper/2203.15266/) (2022)
- [DynaMITe: Dynamic Query Bootstrapping for Multi-object Interactive Segmentation Transformer](https://mlsanity.com/paper/2304.06668/) (2023)
- [Boundary-aware Supervoxel-level Iteratively Refined Interactive 3D Image Segmentation with Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2303.10692/) (2023)
- [DeepEdit: Deep Editable Learning for Interactive Segmentation of 3D Medical Images](https://mlsanity.com/paper/2305.10655/) (2023)
- [Probabilistic Attention for Interactive Segmentation](https://mlsanity.com/paper/2106.15338/) (2021)
- [ScribbleSeg: Scribble-based Interactive Image Segmentation](https://mlsanity.com/paper/2303.11320/) (2023)
- [Scaling up instance annotation via label propagation](https://mlsanity.com/paper/2110.02277/) (2021)
- [Adapting the Segment Anything Model During Usage in Novel Situations](https://mlsanity.com/paper/2404.08421/) (2024)
- [Giving Commands to a Self-Driving Car: How to Deal with Uncertain Situations?](https://mlsanity.com/paper/2106.04232/) (2021)
- [Estimating Generic 3D Room Structures from 2D Annotations](https://mlsanity.com/paper/2306.09077/) (2023)
- [Efficient Human-in-the-loop System for Guiding DNNs Attention](https://mlsanity.com/paper/2206.05981/) (2022)
- [RClicks: Realistic Click Simulation for Benchmarking Interactive Segmentation](https://mlsanity.com/paper/2410.11722/) (2024)
- [Contour-based Interactive Segmentation](https://mlsanity.com/paper/2302.06353/) (2023)
- [Rethinking Interactive Image Segmentation: Feature Space Annotation](https://mlsanity.com/paper/2101.04378/) (2021)
- [Clicking Matters:Towards Interactive Human Parsing](https://mlsanity.com/paper/2111.06162/) (2021)
- [Learning from Exemplars for Interactive Image Segmentation](https://mlsanity.com/paper/2406.11472/) (2024)
- [Interactive Occlusion Boundary Estimation through Exploitation of Synthetic Data](https://mlsanity.com/paper/2408.15038/) (2024)
- [Evaluating and Improving Interactions with Hazy Oracles](https://mlsanity.com/paper/2110.10206/) (2021)
- [How Low Can We Go? Pixel Annotation for Semantic Segmentation](https://mlsanity.com/paper/2201.10448/) (2022)
- [Cascaded Sparse Feature Propagation Network for Interactive Segmentation](https://mlsanity.com/paper/2203.05145/) (2022)
- [Connectivity-constrained Interactive Panoptic Segmentation](https://mlsanity.com/paper/2212.06756/) (2022)
- [Textureless Object Recognition: An Edge-based Approach](https://mlsanity.com/paper/2403.06107/) (2024)

## Abstract (arXiv)

We address interactive full image annotation, where the goal is to accurately segment all object and stuff regions in an image. We propose an interactive, scribble-based annotation framework which operates on the whole image to produce segmentations for all regions. This enables sharing scribble corrections across regions, and allows the annotator to focus on the largest errors made by the machine across the whole image. To realize this, we adapt Mask-RCNN into a fast interactive segmentation framework and introduce an instance-aware loss measured at the pixel-level in the full image canvas, which lets predictions for nearby regions properly compete for space. Finally, we compare to interactive single object segmentation on the COCO panoptic dataset. We demonstrate that our interactive full image segmentation approach leads to a 5% IoU gain, reaching 90% IoU at a budget of four extreme clicks and four corrective scribbles per region.