# Pixelwise Instance Segmentation with a Dynamically Instantiated Network

arXiv: https://arxiv.org/abs/1704.02386 · HTML: https://mlsanity.com/paper/1704.02386/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2017.

## Built on by (most cited first, 21 of 21)

- [MSeg: A Composite Dataset for Multi-domain Semantic Segmentation](https://mlsanity.com/paper/2112.13762/) (2021)
- [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers](https://mlsanity.com/paper/2103.12340/) (2021)
- [Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling](https://mlsanity.com/paper/2111.12698/) (2021)
- [Look Before You Match: Instance Understanding Matters in Video Object Segmentation](https://mlsanity.com/paper/2212.06826/) (2022)
- [Instance and Panoptic Segmentation Using Conditional Convolutions](https://mlsanity.com/paper/2102.03026/) (2021)
- [Learning Equivariant Segmentation with Instance-Unique Querying](https://mlsanity.com/paper/2210.00911/) (2022)
- [Panoramic Panoptic Segmentation: Insights Into Surrounding Parsing for Mobile Agents via Unsupervised Contrastive Learning](https://mlsanity.com/paper/2206.10711/) (2022)
- [Instance segmentation of fallen trees in aerial color infrared imagery using active multi-contour evolution with fully convolutional network-based intensity priors](https://mlsanity.com/paper/2105.01998/) (2021)
- [Open-world Instance Segmentation: Top-down Learning with Bottom-up Supervision](https://mlsanity.com/paper/2303.05503/) (2023)
- [Semantic Attention and Scale Complementary Network for Instance Segmentation in Remote Sensing Images](https://mlsanity.com/paper/2107.11758/) (2021)
- [Occlusion-Aware Instance Segmentation via BiLayer Network Architectures](https://mlsanity.com/paper/2208.04438/) (2022)
- [Hierarchical Lov'asz Embeddings for Proposal-free Panoptic Segmentation](https://mlsanity.com/paper/2106.04555/) (2021)
- [Weakly Supervised Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/2101.00667/) (2021)
- [MARS: Mask Attention Refinement with Sequential Quadtree Nodes for Car Damage Instance Segmentation](https://mlsanity.com/paper/2305.04743/) (2023)
- [Instance Segmentation with Cross-Modal Consistency](https://mlsanity.com/paper/2210.08113/) (2022)
- [Distributional Instance Segmentation: Modeling Uncertainty and High Confidence Predictions with Latent-MaskRCNN](https://mlsanity.com/paper/2305.01910/) (2023)
- [Deep-Learning-Assisted Analysis of Cataract Surgery Videos](https://mlsanity.com/paper/2312.05900/) (2023)
- [RailYolact -- A Yolact Focused on edge for Real-Time Rail Segmentation](https://mlsanity.com/paper/2410.09612/) (2024)
- [Towards Adaptive Open-Set Object Detection via Category-Level Collaboration Knowledge Mining](https://mlsanity.com/paper/2604.11195/) (2026)
- [Multi-View In-Cabin Monitoring System for Public Transport Vehicles](https://mlsanity.com/paper/2606.11739/) (2026)
- [Hybrid supervised instance segmentation by learning label noise suppression](https://mlsanity.com/paper/2610.20412/) (2026)

## Abstract (arXiv)

Semantic segmentation and object detection research have recently achieved rapid progress. However, the former task has no notion of different instances of the same object, and the latter operates at a coarse, bounding-box level. We propose an Instance Segmentation system that produces a segmentation map where each pixel is assigned an object class and instance identity label. Most approaches adapt object detectors to produce segments instead of boxes. In contrast, our method is based on an initial semantic segmentation module, which feeds into an instance subnetwork. This subnetwork uses the initial category-level segmentation, along with cues from the output of an object detector, within an end-to-end CRF to predict instances. This part of our model is dynamically instantiated to produce a variable number of instances per image. Our end-to-end approach requires no post-processing and considers the image holistically, instead of processing independent proposals. Therefore, unlike some related work, a pixel cannot belong to multiple instances. Furthermore, far more precise segmentations are achieved, as shown by our state-of-the-art results (particularly at high IoU thresholds) on the Pascal VOC and Cityscapes datasets.