# Indoor Semantic Segmentation using depth information

arXiv: https://arxiv.org/abs/1301.3572 · HTML: https://mlsanity.com/paper/1301.3572/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2013.

## Built on by (most cited first, 30 of 56)

- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Sapiens: Foundation for Human Vision Models](https://mlsanity.com/paper/2408.12569/) (2024)
- [EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI](https://mlsanity.com/paper/2312.16170/) (2023)
- [RotoGrad: Gradient Homogenization in Multitask Learning](https://mlsanity.com/paper/2103.02631/) (2021)
- [Recon: Reducing Conflicting Gradients from the Root for Multi-Task Learning](https://mlsanity.com/paper/2302.11289/) (2023)
- [Deep Learning Based 3D Segmentation: A Survey](https://mlsanity.com/paper/2103.05423/) (2021)
- [Learning Deep Multimodal Feature Representation with Asymmetric Multi-layer Fusion](https://mlsanity.com/paper/2108.05009/) (2021)
- [3D VSG: Long-term Semantic Scene Change Prediction through 3D Variable Scene Graphs](https://mlsanity.com/paper/2209.07896/) (2022)
- [OmniControlNet: Dual-stage Integration for Conditional Image Generation](https://mlsanity.com/paper/2406.05871/) (2024)
- [AutoMTL: A Programming Framework for Automating Efficient Multi-Task Learning](https://mlsanity.com/paper/2110.13076/) (2021)
- [CaRTS: Causality-driven Robot Tool Segmentation from Vision and Kinematics Data](https://mlsanity.com/paper/2203.09475/) (2022)
- [Near, far: Patch-ordering enhances vision foundation models' scene understanding](https://mlsanity.com/paper/2408.11054/) (2024)
- [Proactive Gradient Conflict Mitigation in Multi-Task Learning: A Sparse Training Perspective](https://mlsanity.com/paper/2411.18615/) (2024)
- [Towards a Computed-Aided Diagnosis System in Colonoscopy: Automatic Polyp Segmentation Using Convolution Neural Networks](https://mlsanity.com/paper/2101.06040/) (2021)
- [Deep Learning for Scene Classification: A Survey](https://mlsanity.com/paper/2101.10531/) (2021)
- [Robust Double-Encoder Network for RGB-D Panoptic Segmentation](https://mlsanity.com/paper/2210.02834/) (2022)
- [Occupancy World Model for Robots](https://mlsanity.com/paper/2505.05512/) (2025)
- [Pixel-aligned RGB-NIR Stereo Imaging and Dataset for Robot Vision](https://mlsanity.com/paper/2411.18025/) (2024)
- [Modality-Incremental Learning with Disjoint Relevance Mapping Networks for Image-based Semantic Segmentation](https://mlsanity.com/paper/2411.17610/) (2024)
- [Helvipad: A Real-World Dataset for Omnidirectional Stereo Depth Estimation](https://mlsanity.com/paper/2411.18335/) (2024)
- [Multi-Modal RGB-D Scene Recognition Across Domains](https://mlsanity.com/paper/2103.14672/) (2021)
- [Deep feature selection-and-fusion for RGB-D semantic segmentation](https://mlsanity.com/paper/2105.04102/) (2021)
- [QueSTMaps: Queryable Semantic Topological Maps for 3D Scene Understanding](https://mlsanity.com/paper/2404.06442/) (2024)
- [Multi-Task Label Discovery via Hierarchical Task Tokens for Partially Annotated Dense Predictions](https://mlsanity.com/paper/2411.18823/) (2024)
- [Revisiting Cross-Modal Knowledge Distillation: A Disentanglement Approach for RGBD Semantic Segmentation](https://mlsanity.com/paper/2505.24361/) (2025)
- [DCL: Differential Contrastive Learning for Geometry-Aware Depth Synthesis](https://mlsanity.com/paper/2107.13087/) (2021)
- [DepthFormer: Multimodal Positional Encodings and Cross-Input Attention for Transformer-Based Segmentation Networks](https://mlsanity.com/paper/2211.04188/) (2022)
- [Learning Task-preferred Inference Routes for Gradient De-conflict in Multi-output DNNs](https://mlsanity.com/paper/2305.19844/) (2023)
- [Multi-View Representation is What You Need for Point-Cloud Pre-Training](https://mlsanity.com/paper/2306.02558/) (2023)
- [Multi-Task Learning with Multi-Task Optimization](https://mlsanity.com/paper/2403.16162/) (2024)

## Abstract (arXiv)

This work addresses multi-class segmentation of indoor scenes with RGB-D inputs. While this area of research has gained much attention recently, most works still rely on hand-crafted features. In contrast, we apply a multiscale convolutional network to learn features directly from the images and the depth information. We obtain state-of-the-art on the NYU-v2 depth dataset with an accuracy of 64.5%. We illustrate the labeling of indoor scenes in videos sequences that could be processed in real-time using appropriate hardware such as an FPGA.