# Accelerating the creation of instance segmentation training sets through bounding box annotation

arXiv: https://arxiv.org/abs/2205.11563 · HTML: https://mlsanity.com/paper/2205.11563/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 24 of 24)

- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [Conditional Convolutions for Instance Segmentation](https://mlsanity.com/paper/2003.05664/) (2020)
- [The VIA Annotation Software for Images, Audio and Video](https://mlsanity.com/paper/1904.10699/) (2019)
- [BoxInst: High-Performance Instance Segmentation with Box Annotations](https://mlsanity.com/paper/2012.02310/) (2020)
- [Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation](https://mlsanity.com/paper/1706.04737/) (2017)
- [Deep Extreme Cut: From Extreme Points to Object Segmentation](https://mlsanity.com/paper/1711.09081/) (2017)
- [Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++](https://mlsanity.com/paper/1803.09693/) (2018)
- [Large-scale interactive object segmentation with human annotators](https://mlsanity.com/paper/1903.10830/) (2019)
- [Fast Interactive Object Annotation with Curve-GCN](https://mlsanity.com/paper/1903.06874/) (2019)
- [Cost-Effective Active Learning for Melanoma Segmentation](https://mlsanity.com/paper/1711.09168/) (2017)
- [Devil is in the Edges: Learning Semantic Boundaries from Noisy Annotations](https://mlsanity.com/paper/1904.07934/) (2019)
- [Cost-Sensitive Active Learning for Intracranial Hemorrhage Detection](https://mlsanity.com/paper/1809.02882/) (2018)
- [Do We Need More Training Data?](https://mlsanity.com/paper/1503.01508/) (2015)
- [Beyond Fixed Grid: Learning Geometric Image Representation with a Deformable Grid](https://mlsanity.com/paper/2008.09269/) (2020)
- [Weakly Supervised Vessel Segmentation in X-ray Angiograms by Self-Paced Learning from Noisy Labels with Suggestive Annotation](https://mlsanity.com/paper/2005.13366/) (2020)
- [ScribbleBox: Interactive Annotation Framework for Video Object Segmentation](https://mlsanity.com/paper/2008.09721/) (2020)
- [Hard Pixel Mining for Depth Privileged Semantic Segmentation](https://mlsanity.com/paper/1906.11437/) (2019)
- [Scaling up instance annotation via label propagation](https://mlsanity.com/paper/2110.02277/) (2021)
- [Coarse-to-Fine Annotation Enrichment for Semantic Segmentation Learning](https://mlsanity.com/paper/1808.07209/) (2018)
- [Mask-guided sample selection for Semi-Supervised Instance Segmentation](https://mlsanity.com/paper/2008.11073/) (2020)
- [Discriminative and Efficient Label Propagation on Complementary Graphs for Multi-Object Tracking](https://mlsanity.com/paper/1504.01124/) (2015)

## Built on by (most cited first, 1 of 1)

- [DeepSportradar-v1: Computer Vision Dataset for Sports Understanding with High Quality Annotations](https://mlsanity.com/paper/2208.08190/) (2022)

## Abstract (arXiv)

Collecting image annotations remains a significant burden when deploying CNN in a specific applicative context. This is especially the case when the annotation consists in binary masks covering object instances. Our work proposes to delineate instances in three steps, based on a semi-automatic approach: (1) the extreme points of an object (left-most, right-most, top, bottom pixels) are manually defined, thereby providing the object bounding-box, (2) a universal automatic segmentation tool like Deep Extreme Cut is used to turn the bounded object into a segmentation mask that matches the extreme points; and (3) the predicted mask is manually corrected. Various strategies are then investigated to balance the human manual annotation resources between bounding-box definition and mask correction, including when the correction of instance masks is prioritized based on their overlap with other instance bounding-boxes, or the outcome of an instance segmentation model trained on a partially annotated dataset. Our experimental study considers a teamsport player segmentation task, and measures how the accuracy of the Panoptic-Deeplab instance segmentation model depends on the human annotation resources allocation strategy. It reveals that the sole definition of extreme points results in a model accuracy that would require up to 10 times more resources if the masks were defined through fully manual delineation of instances. When targeting higher accuracies, prioritizing the mask correction among the training set instances is also shown to save up to 80% of correction annotation resources compared to a systematic frame by frame correction of instances, for a same trained instance segmentation model accuracy.