# Deep Variational Instance Segmentation

arXiv: https://arxiv.org/abs/2007.11576 · HTML: https://mlsanity.com/paper/2007.11576/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jul 2020.

## Built on by (most cited first, 7 of 7)

- [ISTR: End-to-End Instance Segmentation with Transformers](https://mlsanity.com/paper/2105.00637/) (2021)
- [Box-supervised Instance Segmentation with Level Set Evolution](https://mlsanity.com/paper/2207.09055/) (2022)
- [Box2Mask: Box-supervised Instance Segmentation via Level-set Evolution](https://mlsanity.com/paper/2212.01579/) (2022)
- [Pseudo-label Alignment for Semi-supervised Instance Segmentation](https://mlsanity.com/paper/2308.05359/) (2023)
- [VideoClick: Video Object Segmentation with a Single Click](https://mlsanity.com/paper/2101.06545/) (2021)
- [Maximal Cliques on Multi-Frame Proposal Graph for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2301.12352/) (2023)
- [Neural TMDlayer: Modeling Instantaneous flow of features via SDE Generators](https://mlsanity.com/paper/2108.08891/) (2021)

## Abstract (arXiv)

Instance Segmentation, which seeks to obtain both class and instance labels for each pixel in the input image, is a challenging task in computer vision. State-of-the-art algorithms often employ two separate stages, the first one generating object proposals and the second one recognizing and refining the boundaries. Further, proposals are usually based on detectors such as faster R-CNN which search for boxes in the entire image exhaustively. In this paper, we propose a novel algorithm that directly utilizes a fully convolutional network (FCN) to predict instance labels. Specifically, we propose a variational relaxation of instance segmentation as minimizing an optimization functional for a piecewise-constant segmentation problem, which can be used to train an FCN end-to-end. It extends the classical Mumford-Shah variational segmentation problem to be able to handle permutation-invariant labels in the ground truth of instance segmentation. Experiments on PASCAL VOC 2012, Semantic Boundaries dataset(SBD), and the MSCOCO 2017 dataset show that the proposed approach efficiently tackle the instance segmentation task. The source code and trained models will be released with the paper.