# Weakly Supervised Instance Segmentation by Learning Annotation Consistent Instances

arXiv: https://arxiv.org/abs/2007.09397 · HTML: https://mlsanity.com/paper/2007.09397/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2020.

## Built on by (most cited first, 30 of 32)

- [BBAM: Bounding Box Attribution Map for Weakly Supervised Semantic and Instance Segmentation](https://mlsanity.com/paper/2103.08907/) (2021)
- [Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling](https://mlsanity.com/paper/2111.12698/) (2021)
- [Pointly-Supervised Instance Segmentation](https://mlsanity.com/paper/2104.06404/) (2021)
- [DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision](https://mlsanity.com/paper/2105.06464/) (2021)
- [Box-supervised Instance Segmentation with Level Set Evolution](https://mlsanity.com/paper/2207.09055/) (2022)
- [Fully Convolutional Networks for Panoptic Segmentation with Point-based Supervision](https://mlsanity.com/paper/2108.07682/) (2021)
- [BoxTeacher: Exploring High-Quality Pseudo Labels for Weakly Supervised Instance Segmentation](https://mlsanity.com/paper/2210.05174/) (2022)
- [Beyond Semantic to Instance Segmentation: Weakly-Supervised Instance Segmentation via Semantic Knowledge Transfer and Self-Refinement](https://mlsanity.com/paper/2109.09477/) (2021)
- [Box2Mask: Box-supervised Instance Segmentation via Level-set Evolution](https://mlsanity.com/paper/2212.01579/) (2022)
- [The Devil is in the Points: Weakly Semi-Supervised Instance Segmentation via Point-Guided Mask Representation](https://mlsanity.com/paper/2303.15062/) (2023)
- [Weakly-supervised Instance Segmentation via Class-agnostic Learning with Salient Images](https://mlsanity.com/paper/2104.01526/) (2021)
- [Weakly Supervised Instance Segmentation for Videos with Temporal Mask Consistency](https://mlsanity.com/paper/2103.12886/) (2021)
- [Semantic-aware SAM for Point-Prompted Instance Segmentation](https://mlsanity.com/paper/2312.15895/) (2023)
- [Active Pointly-Supervised Instance Segmentation](https://mlsanity.com/paper/2207.11493/) (2022)
- [Solve the Puzzle of Instance Segmentation in Videos: A Weakly Supervised Framework with Spatio-Temporal Collaboration](https://mlsanity.com/paper/2212.07592/) (2022)
- [DG-Labeler and DGL-MOTS Dataset: Boost the Autonomous Driving Perception](https://mlsanity.com/paper/2110.07790/) (2021)
- [WeakSAM: Segment Anything Meets Weakly-supervised Instance-level Recognition](https://mlsanity.com/paper/2402.14812/) (2024)
- [LWSIS: LiDAR-guided Weakly Supervised Instance Segmentation for Autonomous Driving](https://mlsanity.com/paper/2212.03504/) (2022)
- [BLADE: Box-Level Supervised Amodal Segmentation through Directed Expansion](https://mlsanity.com/paper/2401.01642/) (2024)
- [Leveraging Large-Scale Pretrained Vision Foundation Models for Label-Efficient 3D Point Cloud Segmentation](https://mlsanity.com/paper/2311.01989/) (2023)
- [Complete Instances Mining for Weakly Supervised Instance Segmentation](https://mlsanity.com/paper/2402.07633/) (2024)
- [Cyclic Learning: Bridging Image-level Labels and Nuclei Instance Segmentation](https://mlsanity.com/paper/2306.02691/) (2023)
- [What is Point Supervision Worth in Video Instance Segmentation?](https://mlsanity.com/paper/2404.01990/) (2024)
- [Collaborative Propagation on Multiple Instance Graphs for 3D Instance Segmentation with Single-point Supervision](https://mlsanity.com/paper/2208.05110/) (2022)
- [EM-Paste: EM-guided Cut-Paste with DALL-E Augmentation for Image-level Weakly Supervised Instance Segmentation](https://mlsanity.com/paper/2212.07629/) (2022)
- [Weakly Supervised 3D Instance Segmentation without Instance-level Annotations](https://mlsanity.com/paper/2308.01721/) (2023)
- [Extreme Point Supervised Instance Segmentation](https://mlsanity.com/paper/2405.20729/) (2024)
- [Boosting Box-supervised Instance Segmentation with Pseudo Depth](https://mlsanity.com/paper/2403.01214/) (2024)
- [Towards Optimal Heterogeneous Client Sampling in Multi-Model Federated Learning](https://mlsanity.com/paper/2504.05138/) (2025)
- [P2Seg: Pointly-supervised Segmentation via Mutual Distillation](https://mlsanity.com/paper/2401.09709/) (2024)

## Abstract (arXiv)

Recent approaches for weakly supervised instance segmentations depend on two components: (i) a pseudo label generation model that provides instances which are consistent with a given annotation; and (ii) an instance segmentation model, which is trained in a supervised manner using the pseudo labels as ground-truth. Unlike previous approaches, we explicitly model the uncertainty in the pseudo label generation process using a conditional distribution. The samples drawn from our conditional distribution provide accurate pseudo labels due to the use of semantic class aware unary terms, boundary aware pairwise smoothness terms, and annotation aware higher order terms. Furthermore, we represent the instance segmentation model as an annotation agnostic prediction distribution. In contrast to previous methods, our representation allows us to define a joint probabilistic learning objective that minimizes the dissimilarity between the two distributions. Our approach achieves state of the art results on the PASCAL VOC 2012 data set, outperforming the best baseline by 4.2% mAP@0.5 and 4.8% mAP@0.75.