# Pixel-level Encoding and Depth Layering for Instance-level Semantic Labeling

arXiv: https://arxiv.org/abs/1604.05096 · HTML: https://mlsanity.com/paper/1604.05096/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Apr 2016.

## Built on by (most cited first, 10 of 10)

- [LaneAF: Robust Multi-Lane Detection with Affinity Fields](https://mlsanity.com/paper/2103.12040/) (2021)
- [CalibNet: Dual-branch Cross-modal Calibration for RGB-D Salient Instance Segmentation](https://mlsanity.com/paper/2307.08098/) (2023)
- [VideoClick: Video Object Segmentation with a Single Click](https://mlsanity.com/paper/2101.06545/) (2021)
- [Exploiting Depth Information for Wildlife Monitoring](https://mlsanity.com/paper/2102.05607/) (2021)
- [Real-time Instance Segmentation with Discriminative Orientation Maps](https://mlsanity.com/paper/2106.12204/) (2021)
- [Hyperspherical Embedding for Point Cloud Completion](https://mlsanity.com/paper/2307.05634/) (2023)
- [Hybridnet for depth estimation and semantic segmentation](https://mlsanity.com/paper/2402.06539/) (2024)
- [Leveraging convergence behavior to balance conflicting tasks in multi-task learning](https://mlsanity.com/paper/2204.06698/) (2022)
- [Nuclei panoptic segmentation and composition regression with multi-task deep neural networks](https://mlsanity.com/paper/2202.11804/) (2022)
- [Multi-scale attention-based instance segmentation for measuring crystals with large size variation](https://mlsanity.com/paper/2401.03939/) (2024)

## Abstract (arXiv)

Recent approaches for instance-aware semantic labeling have augmented convolutional neural networks (CNNs) with complex multi-task architectures or computationally expensive graphical models. We present a method that leverages a fully convolutional network (FCN) to predict semantic labels, depth and an instance-based encoding using each pixel's direction towards its corresponding instance center. Subsequently, we apply low-level computer vision techniques to generate state-of-the-art instance segmentation on the street scene datasets KITTI and Cityscapes. Our approach outperforms existing works by a large margin and can additionally predict absolute distances of individual instances from a monocular image as well as a pixel-level semantic labeling.