# Active Pointly-Supervised Instance Segmentation

arXiv: https://arxiv.org/abs/2207.11493 · HTML: https://mlsanity.com/paper/2207.11493/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 63 references that are in the map.
- First posted on arXiv 23 Jul 2022.

## Builds on (most cited first, 30 of 48)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [Conditional Convolutions for Instance Segmentation](https://mlsanity.com/paper/2003.05664/) (2020)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [Mask Scoring R-CNN](https://mlsanity.com/paper/1903.00241/) (2019)
- [Simple Does It: Weakly Supervised Instance and Semantic Segmentation](https://mlsanity.com/paper/1603.07485/) (2016)
- [Cost-Effective Active Learning for Deep Image Classification](https://mlsanity.com/paper/1701.03551/) (2017)
- [Deep Interactive Object Selection](https://mlsanity.com/paper/1603.04042/) (2016)
- [Instances as Queries](https://mlsanity.com/paper/2105.01928/) (2021)
- [BoxInst: High-Performance Instance Segmentation with Box Annotations](https://mlsanity.com/paper/2012.02310/) (2020)
- [Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation](https://mlsanity.com/paper/1706.04737/) (2017)
- [Contextual Diversity for Active Learning](https://mlsanity.com/paper/2008.05723/) (2020)
- [Deep Extreme Cut: From Extreme Points to Object Segmentation](https://mlsanity.com/paper/1711.09081/) (2017)
- [An Analysis of Scale Invariance in Object Detection - SNIP](https://mlsanity.com/paper/1711.08189/) (2017)
- [Large-scale interactive object segmentation with human annotators](https://mlsanity.com/paper/1903.10830/) (2019)
- [AutoAssign: Differentiable Label Assignment for Dense Object Detection](https://mlsanity.com/paper/2007.03496/) (2020)
- [Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation](https://mlsanity.com/paper/1503.00848/) (2015)
- [Influence Selection for Active Learning](https://mlsanity.com/paper/2108.09331/) (2021)
- [Pointly-Supervised Instance Segmentation](https://mlsanity.com/paper/2104.06404/) (2021)
- [Extreme clicking for efficient object annotation](https://mlsanity.com/paper/1708.02750/) (2017)
- [Active Learning for Deep Object Detection via Probabilistic Modeling](https://mlsanity.com/paper/2103.16130/) (2021)

## Built on by (most cited first, 15 of 15)

- [A comprehensive survey on deep active learning in medical image analysis](https://mlsanity.com/paper/2310.14230/) (2023)
- [Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport](https://mlsanity.com/paper/2308.01779/) (2023)
- [From colouring-in to pointillism: revisiting semantic segmentation supervision](https://mlsanity.com/paper/2210.14142/) (2022)
- [Box2Mask: Box-supervised Instance Segmentation via Level-set Evolution](https://mlsanity.com/paper/2212.01579/) (2022)
- [Label-efficient Segmentation via Affinity Propagation](https://mlsanity.com/paper/2310.10533/) (2023)
- [GaPro: Box-Supervised 3D Point Cloud Instance Segmentation Using Gaussian Processes as Pseudo Labelers](https://mlsanity.com/paper/2307.13251/) (2023)
- [Finding Meaning in Points: Weakly Supervised Semantic Segmentation for Event Cameras](https://mlsanity.com/paper/2407.11216/) (2024)
- [Active Coarse-to-Fine Segmentation of Moveable Parts from Real Images](https://mlsanity.com/paper/2303.11530/) (2023)
- [Extreme Point Supervised Instance Segmentation](https://mlsanity.com/paper/2405.20729/) (2024)
- [PVPUFormer: Probabilistic Visual Prompt Unified Transformer for Interactive Image Segmentation](https://mlsanity.com/paper/2306.06656/) (2023)
- [Boosting Box-supervised Instance Segmentation with Pseudo Depth](https://mlsanity.com/paper/2403.01214/) (2024)
- [Two-Step Active Learning for Instance Segmentation with Uncertainty and Diversity Sampling](https://mlsanity.com/paper/2309.16139/) (2023)
- [P2Object: Single Point Supervised Object Detection and Instance Segmentation](https://mlsanity.com/paper/2504.07813/) (2025)
- [ReSAM: Refine, Requery, and Reinforce: Self-Prompting Point-Supervised Segmentation for Remote Sensing Images](https://mlsanity.com/paper/2511.21606/) (2025)
- [HiLLIE: Human-in-the-Loop Training for Low-Light Image Enhancement](https://mlsanity.com/paper/2505.02134/) (2025)

## Abstract (arXiv)

The requirement of expensive annotations is a major burden for training a well-performed instance segmentation model. In this paper, we present an economic active learning setting, named active pointly-supervised instance segmentation (APIS), which starts with box-level annotations and iteratively samples a point within the box and asks if it falls on the object. The key of APIS is to find the most desirable points to maximize the segmentation accuracy with limited annotation budgets. We formulate this setting and propose several uncertainty-based sampling strategies. The model developed with these strategies yields consistent performance gain on the challenging MS-COCO dataset, compared against other learning strategies. The results suggest that APIS, integrating the advantages of active learning and point-based supervision, is an effective learning paradigm for label-efficient instance segmentation.