# InstanceCut: from Edges to Instances with MultiCut

arXiv: https://arxiv.org/abs/1611.08272 · HTML: https://mlsanity.com/paper/1611.08272/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2016.

## Built on by (most cited first, 30 of 49)

- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [K-Net: Towards Unified Image Segmentation](https://mlsanity.com/paper/2106.14855/) (2021)
- [Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP](https://mlsanity.com/paper/2308.02487/) (2023)
- [RTMDet: An Empirical Study of Designing Real-Time Object Detectors](https://mlsanity.com/paper/2212.07784/) (2022)
- [MSeg: A Composite Dataset for Multi-domain Semantic Segmentation](https://mlsanity.com/paper/2112.13762/) (2021)
- [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers](https://mlsanity.com/paper/2103.12340/) (2021)
- [Pointly-Supervised Instance Segmentation](https://mlsanity.com/paper/2104.06404/) (2021)
- [Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation](https://mlsanity.com/paper/2104.05239/) (2021)
- [Spatial Feature Calibration and Temporal Fusion for Effective One-stage Video Instance Segmentation](https://mlsanity.com/paper/2104.05606/) (2021)
- [In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2408.04961/) (2024)
- [CLUSTSEG: Clustering for Universal Segmentation](https://mlsanity.com/paper/2305.02187/) (2023)
- [Semantic Image Segmentation: Two Decades of Research](https://mlsanity.com/paper/2302.06378/) (2023)
- [A Weakly Supervised Amodal Segmenter with Boundary Uncertainty Estimation](https://mlsanity.com/paper/2108.09897/) (2021)
- [Camouflaged Instance Segmentation In-The-Wild: Dataset, Method, and Benchmark Suite](https://mlsanity.com/paper/2103.17123/) (2021)
- [FAPIS: A Few-shot Anchor-free Part-based Instance Segmenter](https://mlsanity.com/paper/2104.00073/) (2021)
- [A^2-FPN: Attention Aggregation based Feature Pyramid Network for Instance Segmentation](https://mlsanity.com/paper/2105.03186/) (2021)
- [SharpContour: A Contour-based Boundary Refinement Approach for Efficient and Accurate Instance Segmentation](https://mlsanity.com/paper/2203.13312/) (2022)
- [Delving into Crispness: Guided Label Refinement for Crisp Edge Detection](https://mlsanity.com/paper/2306.15172/) (2023)
- [Combinatorial Optimization for Panoptic Segmentation: A Fully Differentiable Approach](https://mlsanity.com/paper/2106.03188/) (2021)
- [RAMA: A Rapid Multicut Algorithm on GPU](https://mlsanity.com/paper/2109.01838/) (2021)
- [You Only Learn One Query: Learning Unified Human Query for Single-Stage Multi-Person Multi-Task Human-Centric Perception](https://mlsanity.com/paper/2312.05525/) (2023)
- [VideoClick: Video Object Segmentation with a Single Click](https://mlsanity.com/paper/2101.06545/) (2021)
- [PointScatter: Point Set Representation for Tubular Structure Extraction](https://mlsanity.com/paper/2209.05774/) (2022)
- [Unsupervised domain adaptation for clinician pose estimation and instance segmentation in the operating room](https://mlsanity.com/paper/2108.11801/) (2021)
- [CenterPoly: real-time instance segmentation using bounding polygons](https://mlsanity.com/paper/2108.08923/) (2021)
- [Segmentation in large-scale cellular electron microscopy with deep learning: A literature survey](https://mlsanity.com/paper/2206.07171/) (2022)
- [Semantic Attention and Scale Complementary Network for Instance Segmentation in Remote Sensing Images](https://mlsanity.com/paper/2107.11758/) (2021)
- [SDTP: Semantic-aware Decoupled Transformer Pyramid for Dense Image Prediction](https://mlsanity.com/paper/2109.08963/) (2021)
- [Occlusion-Aware Instance Segmentation via BiLayer Network Architectures](https://mlsanity.com/paper/2208.04438/) (2022)
- [ClusterFuG: Clustering Fully connected Graphs by Multicut](https://mlsanity.com/paper/2301.12159/) (2023)

## Abstract (arXiv)

This work addresses the task of instance-aware semantic segmentation. Our key motivation is to design a simple method with a new modelling-paradigm, which therefore has a different trade-off between advantages and disadvantages compared to known approaches. Our approach, we term InstanceCut, represents the problem by two output modalities: (i) an instance-agnostic semantic segmentation and (ii) all instance-boundaries. The former is computed from a standard convolutional neural network for semantic segmentation, and the latter is derived from a new instance-aware edge detection model. To reason globally about the optimal partitioning of an image into instances, we combine these two modalities into a novel MultiCut formulation. We evaluate our approach on the challenging CityScapes dataset. Despite the conceptual simplicity of our approach, we achieve the best result among all published methods, and perform particularly well for rare object classes.