# Achieving RGB-D level Segmentation Performance from a Single ToF Camera

arXiv: https://arxiv.org/abs/2306.17636 · HTML: https://mlsanity.com/paper/2306.17636/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 30 Jun 2023.

## Builds on (most cited first, 11 of 11)

- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Multi-Task Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2009.09796/) (2020)
- [ShapeConv: Shape-aware Convolutional Layer for Indoor RGB-D Semantic Segmentation](https://mlsanity.com/paper/2108.10528/) (2021)
- [Depth-aware CNN for RGB-D Segmentation](https://mlsanity.com/paper/1803.06791/) (2018)
- [Semi-supervised Multi-task Learning for Semantics and Depth](https://mlsanity.com/paper/2110.07197/) (2021)
- [TICaM: A Time-of-flight In-car Cabin Monitoring Dataset](https://mlsanity.com/paper/2103.11719/) (2021)
- [TIMo -- A Dataset for Indoor Building Monitoring with a Time-of-Flight Camera](https://mlsanity.com/paper/2108.12196/) (2021)
- [3D Neighborhood Convolution: Learning Depth-Aware Features for RGB-D and RGB Semantic Segmentation](https://mlsanity.com/paper/1910.01460/) (2019)
- [Unsupervised Anomaly Detection from Time-of-Flight Depth Images](https://mlsanity.com/paper/2203.01052/) (2022)
- [Review on Indoor RGB-D Semantic Segmentation with Deep Convolutional Neural Networks](https://mlsanity.com/paper/2105.11925/) (2021)

## Abstract (arXiv)

Depth is a very important modality in computer vision, typically used as complementary information to RGB, provided by RGB-D cameras. In this work, we show that it is possible to obtain the same level of accuracy as RGB-D cameras on a semantic segmentation task using infrared (IR) and depth images from a single Time-of-Flight (ToF) camera. In order to fuse the IR and depth modalities of the ToF camera, we introduce a method utilizing depth-specific convolutions in a multi-task learning framework. In our evaluation on an in-car segmentation dataset, we demonstrate the competitiveness of our method against the more costly RGB-D approaches.