# Weakly-supervised Instance Segmentation via Class-agnostic Learning with Salient Images

arXiv: https://arxiv.org/abs/2104.01526 · HTML: https://mlsanity.com/paper/2104.01526/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 59 references that are in the map.
- First posted on arXiv 4 Apr 2021.

## Builds on (most cited first, 30 of 40)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1604.05144/) (2016)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations](https://mlsanity.com/paper/1904.05044/) (2019)
- [BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1503.01640/) (2015)
- [EGNet:Edge Guidance Network for Salient Object Detection](https://mlsanity.com/paper/1908.08297/) (2019)
- [Simple Does It: Weakly Supervised Instance and Semantic Segmentation](https://mlsanity.com/paper/1603.07485/) (2016)
- [CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning](https://mlsanity.com/paper/1903.02351/) (2019)
- [Cascaded Partial Decoder for Fast and Accurate Salient Object Detection](https://mlsanity.com/paper/1904.08739/) (2019)
- [A Simple Pooling-Based Design for Real-Time Salient Object Detection](https://mlsanity.com/paper/1904.09569/) (2019)
- [Deeply supervised salient object detection with short connections](https://mlsanity.com/paper/1611.04849/) (2016)
- [Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation](https://mlsanity.com/paper/1502.02734/) (2015)
- [BoxInst: High-Performance Instance Segmentation with Box Annotations](https://mlsanity.com/paper/2012.02310/) (2020)
- [Salient Object Detection: A Benchmark](https://mlsanity.com/paper/1501.02741/) (2015)
- [Reverse Attention for Salient Object Detection](https://mlsanity.com/paper/1807.09940/) (2018)
- [Normalized Cut Loss for Weakly-supervised CNN Segmentation](https://mlsanity.com/paper/1804.01346/) (2018)
- [BBAM: Bounding Box Attribution Map for Weakly Supervised Semantic and Instance Segmentation](https://mlsanity.com/paper/2103.08907/) (2021)
- [Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1904.11693/) (2019)
- [Weakly-Supervised Salient Object Detection via Scribble Annotations](https://mlsanity.com/paper/2003.07685/) (2020)
- [Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection](https://mlsanity.com/paper/1708.02001/) (2017)
- [Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation](https://mlsanity.com/paper/1503.00848/) (2015)
- [Salient Object Detection: A Discriminative Regional Feature Integration Approach](https://mlsanity.com/paper/1410.5926/) (2014)
- [Learning to Segment Object Candidates](https://mlsanity.com/paper/1506.06204/) (2015)
- [Weakly Supervised Instance Segmentation using Class Peak Response](https://mlsanity.com/paper/1804.00880/) (2018)
- [Learning to Segment Every Thing](https://mlsanity.com/paper/1711.10370/) (2017)

## Built on by (most cited first, 20 of 20)

- [Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling](https://mlsanity.com/paper/2111.12698/) (2021)
- [A Survey on Label-efficient Deep Image Segmentation: Bridging the Gap between Weak Supervision and Dense Prediction](https://mlsanity.com/paper/2207.01223/) (2022)
- [Box-supervised Instance Segmentation with Level Set Evolution](https://mlsanity.com/paper/2207.09055/) (2022)
- [BoxTeacher: Exploring High-Quality Pseudo Labels for Weakly Supervised Instance Segmentation](https://mlsanity.com/paper/2210.05174/) (2022)
- [Box2Mask: Box-supervised Instance Segmentation via Level-set Evolution](https://mlsanity.com/paper/2212.01579/) (2022)
- [BoxSnake: Polygonal Instance Segmentation with Box Supervision](https://mlsanity.com/paper/2303.11630/) (2023)
- [SIM: Semantic-aware Instance Mask Generation for Box-Supervised Instance Segmentation](https://mlsanity.com/paper/2303.08578/) (2023)
- [WeakSAM: Segment Anything Meets Weakly-supervised Instance-level Recognition](https://mlsanity.com/paper/2402.14812/) (2024)
- [LWSIS: LiDAR-guided Weakly Supervised Instance Segmentation for Autonomous Driving](https://mlsanity.com/paper/2212.03504/) (2022)
- [GETAM: Gradient-weighted Element-wise Transformer Attention Map for Weakly-supervised Semantic segmentation](https://mlsanity.com/paper/2112.02841/) (2021)
- [DAWN: Domain-Adaptive Weakly Supervised Nuclei Segmentation via Cross-Task Interactions](https://mlsanity.com/paper/2404.14956/) (2024)
- [MWSIS: Multimodal Weakly Supervised Instance Segmentation with 2D Box Annotations for Autonomous Driving](https://mlsanity.com/paper/2312.06988/) (2023)
- [ElC-OIS: Ellipsoidal Clustering for Open-World Instance Segmentation on LiDAR Data](https://mlsanity.com/paper/2303.04351/) (2023)
- [Extreme Point Supervised Instance Segmentation](https://mlsanity.com/paper/2405.20729/) (2024)
- [ENInst: Enhancing Weakly-supervised Low-shot Instance Segmentation](https://mlsanity.com/paper/2302.09765/) (2023)
- [Boosting Box-supervised Instance Segmentation with Pseudo Depth](https://mlsanity.com/paper/2403.01214/) (2024)
- [PM-VIS: High-Performance Box-Supervised Video Instance Segmentation](https://mlsanity.com/paper/2404.13863/) (2024)
- [Towards Optimal Heterogeneous Client Sampling in Multi-Model Federated Learning](https://mlsanity.com/paper/2504.05138/) (2025)
- [Deep Level Set for Box-supervised Instance Segmentation in Aerial Images](https://mlsanity.com/paper/2112.03451/) (2021)
- [BoxVIS: Video Instance Segmentation with Box Annotations](https://mlsanity.com/paper/2303.14618/) (2023)

## Abstract (arXiv)

Humans have a strong class-agnostic object segmentation ability and can outline boundaries of unknown objects precisely, which motivates us to propose a box-supervised class-agnostic object segmentation (BoxCaseg) based solution for weakly-supervised instance segmentation. The BoxCaseg model is jointly trained using box-supervised images and salient images in a multi-task learning manner. The fine-annotated salient images provide class-agnostic and precise object localization guidance for box-supervised images. The object masks predicted by a pretrained BoxCaseg model are refined via a novel merged and dropped strategy as proxy ground truth to train a Mask R-CNN for weakly-supervised instance segmentation. Only using 7991 salient images, the weakly-supervised Mask R-CNN is on par with fully-supervised Mask R-CNN on PASCAL VOC and significantly outperforms previous state-of-the-art box-supervised instance segmentation methods on COCO. The source code, pretrained models and datasets are available at urlhttps://github.com/hustvl/BoxCaseg.