# Semi-convolutional Operators for Instance Segmentation

arXiv: https://arxiv.org/abs/1807.10712 · HTML: https://mlsanity.com/paper/1807.10712/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jul 2018.

## Built on by (most cited first, 22 of 22)

- [GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement](https://mlsanity.com/paper/2104.09958/) (2021)
- [SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentation](https://mlsanity.com/paper/2103.10284/) (2021)
- [Contrastive Lift: 3D Object Instance Segmentation by Slow-Fast Contrastive Fusion](https://mlsanity.com/paper/2306.04633/) (2023)
- [Convolutional Hough Matching Networks](https://mlsanity.com/paper/2103.16831/) (2021)
- [SOLO: A Simple Framework for Instance Segmentation](https://mlsanity.com/paper/2106.15947/) (2021)
- [Semantic Image Segmentation: Two Decades of Research](https://mlsanity.com/paper/2302.06378/) (2023)
- [Instance and Panoptic Segmentation Using Conditional Convolutions](https://mlsanity.com/paper/2102.03026/) (2021)
- [Dynamic Attention guided Multi-Trajectory Analysis for Single Object Tracking](https://mlsanity.com/paper/2103.16086/) (2021)
- [Kernel Proposal Network for Arbitrary Shape Text Detection](https://mlsanity.com/paper/2203.06410/) (2022)
- [RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks](https://mlsanity.com/paper/2106.15711/) (2021)
- [DynaMITe: Dynamic Query Bootstrapping for Multi-object Interactive Segmentation Transformer](https://mlsanity.com/paper/2304.06668/) (2023)
- [Convolutional Hough Matching Networks for Robust and Efficient Visual Correspondence](https://mlsanity.com/paper/2109.05221/) (2021)
- [3D Instance Segmentation of MVS Buildings](https://mlsanity.com/paper/2112.09902/) (2021)
- [DeepSportLab: a Unified Framework for Ball Detection, Player Instance Segmentation and Pose Estimation in Team Sports Scenes](https://mlsanity.com/paper/2112.00627/) (2021)
- [How Shift Equivariance Impacts Metric Learning for Instance Segmentation](https://mlsanity.com/paper/2101.05846/) (2021)
- [Real-time Instance Segmentation with Discriminative Orientation Maps](https://mlsanity.com/paper/2106.12204/) (2021)
- [ClusteringSDF: Self-Organized Neural Implicit Surfaces for 3D Decomposition](https://mlsanity.com/paper/2403.14619/) (2024)
- [Embedding-based Instance Segmentation in Microscopy](https://mlsanity.com/paper/2101.10033/) (2021)
- [supervised adptive threshold network for instance segmentation](https://mlsanity.com/paper/2106.03450/) (2021)
- [Compositional Scene Modeling with Global Object-Centric Representations](https://mlsanity.com/paper/2211.11500/) (2022)
- [P2Seg: Pointly-supervised Segmentation via Mutual Distillation](https://mlsanity.com/paper/2401.09709/) (2024)
- [Hybrid supervised instance segmentation by learning label noise suppression](https://mlsanity.com/paper/2610.20412/) (2026)

## Abstract (arXiv)

Object detection and instance segmentation are dominated by region-based methods such as Mask RCNN. However, there is a growing interest in reducing these problems to pixel labeling tasks, as the latter could be more efficient, could be integrated seamlessly in image-to-image network architectures as used in many other tasks, and could be more accurate for objects that are not well approximated by bounding boxes. In this paper we show theoretically and empirically that constructing dense pixel embeddings that can separate object instances cannot be easily achieved using convolutional operators. At the same time, we show that simple modifications, which we call semi-convolutional, have a much better chance of succeeding at this task. We use the latter to show a connection to Hough voting as well as to a variant of the bilateral kernel that is spatially steered by a convolutional network. We demonstrate that these operators can also be used to improve approaches such as Mask RCNN, demonstrating better segmentation of complex biological shapes and PASCAL VOC categories than achievable by Mask RCNN alone.