# Large-scale interactive object segmentation with human annotators

arXiv: https://arxiv.org/abs/1903.10830 · HTML: https://mlsanity.com/paper/1903.10830/

## Key facts

- Cited by 110 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2019.

## Built on by (most cited first, 30 of 110)

- [Scaling Robot Learning with Semantically Imagined Experience](https://mlsanity.com/paper/2302.11550/) (2023)
- [Reviving Iterative Training with Mask Guidance for Interactive Segmentation](https://mlsanity.com/paper/2102.06583/) (2021)
- [EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations](https://mlsanity.com/paper/2209.13064/) (2022)
- [MSeg: A Composite Dataset for Multi-domain Semantic Segmentation](https://mlsanity.com/paper/2112.13762/) (2021)
- [Self-Support Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2207.11549/) (2022)
- [Pointly-Supervised Instance Segmentation](https://mlsanity.com/paper/2104.06404/) (2021)
- [Shelf-Supervised Mesh Prediction in the Wild](https://mlsanity.com/paper/2102.06195/) (2021)
- [High-Resolution Image Harmonization via Collaborative Dual Transformations](https://mlsanity.com/paper/2109.06671/) (2021)
- [PASS: An ImageNet replacement for self-supervised pretraining without humans](https://mlsanity.com/paper/2109.13228/) (2021)
- [ScribblePrompt: Fast and Flexible Interactive Segmentation for Any Biomedical Image](https://mlsanity.com/paper/2312.07381/) (2023)
- [MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning](https://mlsanity.com/paper/2104.12166/) (2021)
- [PseudoClick: Interactive Image Segmentation with Click Imitation](https://mlsanity.com/paper/2207.05282/) (2022)
- [EdgeFlow: Achieving Practical Interactive Segmentation with Edge-Guided Flow](https://mlsanity.com/paper/2109.09406/) (2021)
- [ECCV Caption: Correcting False Negatives by Collecting Machine-and-Human-verified Image-Caption Associations for MS-COCO](https://mlsanity.com/paper/2204.03359/) (2022)
- [AGILE3D: Attention Guided Interactive Multi-object 3D Segmentation](https://mlsanity.com/paper/2306.00977/) (2023)
- [DaTaSeg: Taming a Universal Multi-Dataset Multi-Task Segmentation Model](https://mlsanity.com/paper/2306.01736/) (2023)
- [UniGS: Unified Representation for Image Generation and Segmentation](https://mlsanity.com/paper/2312.01985/) (2023)
- [iSegFormer: Interactive Segmentation via Transformers with Application to 3D Knee MR Images](https://mlsanity.com/paper/2112.11325/) (2021)
- [Vision-Language Intelligence: Tasks, Representation Learning, and Large Models](https://mlsanity.com/paper/2203.01922/) (2022)
- [Human-Aided Saliency Maps Improve Generalization of Deep Learning](https://mlsanity.com/paper/2105.03492/) (2021)
- [F-CAM: Full Resolution Class Activation Maps via Guided Parametric Upscaling](https://mlsanity.com/paper/2109.07069/) (2021)
- [Weakly-Supervised Salient Object Detection Using Point Supervision](https://mlsanity.com/paper/2203.11652/) (2022)
- [Bridging the Gap between Classification and Localization for Weakly Supervised Object Localization](https://mlsanity.com/paper/2204.00220/) (2022)
- [A Spacecraft Dataset for Detection, Segmentation and Parts Recognition](https://mlsanity.com/paper/2106.08186/) (2021)
- [Interactive Object Segmentation in 3D Point Clouds](https://mlsanity.com/paper/2204.07183/) (2022)
- [Modulating Pretrained Diffusion Models for Multimodal Image Synthesis](https://mlsanity.com/paper/2302.12764/) (2023)
- [Benchmarking Object Detectors with COCO: A New Path Forward](https://mlsanity.com/paper/2403.18819/) (2024)
- [From colouring-in to pointillism: revisiting semantic segmentation supervision](https://mlsanity.com/paper/2210.14142/) (2022)
- [InterFormer: Real-time Interactive Image Segmentation](https://mlsanity.com/paper/2304.02942/) (2023)
- [Learning the What and How of Annotation in Video Object Segmentation](https://mlsanity.com/paper/2311.04414/) (2023)

## Abstract (arXiv)

Manually annotating object segmentation masks is very time consuming. Interactive object segmentation methods offer a more efficient alternative where a human annotator and a machine segmentation model collaborate. In this paper we make several contributions to interactive segmentation: (1) we systematically explore in simulation the design space of deep interactive segmentation models and report new insights and caveats; (2) we execute a large-scale annotation campaign with real human annotators, producing masks for 2.5M instances on the OpenImages dataset. We plan to release this data publicly, forming the largest existing dataset for instance segmentation. Moreover, by re-annotating part of the COCO dataset, we show that we can produce instance masks 3 times faster than traditional polygon drawing tools while also providing better quality. (3) We present a technique for automatically estimating the quality of the produced masks which exploits indirect signals from the annotation process.