# Bridging Category-level and Instance-level Semantic Image Segmentation

arXiv: https://arxiv.org/abs/1605.06885 · HTML: https://mlsanity.com/paper/1605.06885/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2016.

## Built on by (most cited first, 21 of 21)

- [Discovering Object Masks with Transformers for Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2206.06363/) (2022)
- [Learning Statistical Texture for Semantic Segmentation](https://mlsanity.com/paper/2103.04133/) (2021)
- [Loss Functions in the Era of Semantic Segmentation: A Survey and Outlook](https://mlsanity.com/paper/2312.05391/) (2023)
- [Geometric Unsupervised Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2103.16694/) (2021)
- [AutoLoss: Automated Loss Function Search in Recommendations](https://mlsanity.com/paper/2106.06713/) (2021)
- [AutoLoss-Zero: Searching Loss Functions from Scratch for Generic Tasks](https://mlsanity.com/paper/2103.14026/) (2021)
- [Do We Really Need Dice? The Hidden Region-Size Biases of Segmentation Losses](https://mlsanity.com/paper/2104.08717/) (2021)
- [Revisiting Image Deblurring with an Efficient ConvNet](https://mlsanity.com/paper/2302.02234/) (2023)
- [ORDNet: Capturing Omni-Range Dependencies for Scene Parsing](https://mlsanity.com/paper/2101.03929/) (2021)
- [MUNet: Motion Uncertainty-aware Semi-supervised Video Object Segmentation](https://mlsanity.com/paper/2111.14646/) (2021)
- [Cross Modal Distillation for Flood Extent Mapping](https://mlsanity.com/paper/2302.08180/) (2023)
- [Distributed Semantic Segmentation with Efficient Joint Source and Task Decoding](https://mlsanity.com/paper/2407.11224/) (2024)
- [SemiDAViL: Semi-supervised Domain Adaptation with Vision-Language Guidance for Semantic Segmentation](https://mlsanity.com/paper/2504.06389/) (2025)
- [Automatically Segment the Left Atrium and Scars from LGE-MRIs Using a Boundary-focused nnU-Net](https://mlsanity.com/paper/2304.14071/) (2023)
- [Pixel-wise Modulated Dice Loss for Medical Image Segmentation](https://mlsanity.com/paper/2506.15744/) (2025)
- [Dilated SpineNet for Semantic Segmentation](https://mlsanity.com/paper/2103.12270/) (2021)
- [Adaptive Voxel-Weighted Loss Using L1 Norms in Deep Neural Networks for Detection and Segmentation of Prostate Cancer Lesions in PET/CT Images](https://mlsanity.com/paper/2502.02756/) (2025)
- [Decomposing and Coupling Saliency Map for Lesion Segmentation in Ultrasound Images](https://mlsanity.com/paper/2308.00947/) (2023)
- [Your Image is My Video: Reshaping the Receptive Field via Image-To-Video Differentiable AutoAugmentation and Fusion](https://mlsanity.com/paper/2403.15194/) (2024)
- [Semantic Class Distribution Learning for Debiasing Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2603.05202/) (2026)
- [Rare-Aware Autoencoding: Reconstructing Spatially Imbalanced Data](https://mlsanity.com/paper/2604.02031/) (2026)

## Abstract (arXiv)

We propose an approach to instance-level image segmentation that is built on top of category-level segmentation. Specifically, for each pixel in a semantic category mask, its corresponding instance bounding box is predicted using a deep fully convolutional regression network. Thus it follows a different pipeline to the popular detect-then-segment approaches that first predict instances' bounding boxes, which are the current state-of-the-art in instance segmentation. We show that, by leveraging the strength of our state-of-the-art semantic segmentation models, the proposed method can achieve comparable or even better results to detect-then-segment approaches. We make the following contributions. (i) First, we propose a simple yet effective approach to semantic instance segmentation. (ii) Second, we propose an online bootstrapping method during training, which is critically important for achieving good performance for both semantic category segmentation and instance-level segmentation. (iii) As the performance of semantic category segmentation has a significant impact on the instance-level segmentation, which is the second step of our approach, we train fully convolutional residual networks to achieve the best semantic category segmentation accuracy. On the PASCAL VOC 2012 dataset, we obtain the currently best mean intersection-over-union score of 79.1%. (iv) We also achieve state-of-the-art results for instance-level segmentation.