# Incremental Class Discovery for Semantic Segmentation with RGBD Sensing

arXiv: https://arxiv.org/abs/1907.10008 · HTML: https://mlsanity.com/paper/1907.10008/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2019.

## Built on by (most cited first, 13 of 13)

- [UnScene3D: Unsupervised 3D Instance Segmentation for Indoor Scenes](https://mlsanity.com/paper/2303.14541/) (2023)
- [ILabel: Interactive Neural Scene Labelling](https://mlsanity.com/paper/2111.14637/) (2021)
- [MixReorg: Cross-Modal Mixed Patch Reorganization is a Good Mask Learner for Open-World Semantic Segmentation](https://mlsanity.com/paper/2308.04829/) (2023)
- [Towards Unsupervised Open World Semantic Segmentation](https://mlsanity.com/paper/2201.01073/) (2022)
- [Dual-level Adaptive Self-Labeling for Novel Class Discovery in Point Cloud Segmentation](https://mlsanity.com/paper/2407.12489/) (2024)
- [Learning Geodesic-Aware Local Features from RGB-D Images](https://mlsanity.com/paper/2203.12016/) (2022)
- [SCIM: Simultaneous Clustering, Inference, and Mapping for Open-World Semantic Scene Understanding](https://mlsanity.com/paper/2206.10670/) (2022)
- [Training Semantic Segmentation on Heterogeneous Datasets](https://mlsanity.com/paper/2301.07634/) (2023)
- [Pixel-Level Clustering Network for Unsupervised Image Segmentation](https://mlsanity.com/paper/2310.16234/) (2023)
- [Enhancing Long-Term Person Re-Identification Using Global, Local Body Part, and Head Streams](https://mlsanity.com/paper/2403.02892/) (2024)
- [Detecting Novelties with Empty Classes](https://mlsanity.com/paper/2305.00983/) (2023)
- [NeurNCD: Novel Class Discovery via Implicit Neural Representation](https://mlsanity.com/paper/2506.06412/) (2025)
- [MagicSeg: Open-World Segmentation Pretraining via Counterfactural Diffusion-Based Auto-Generation](https://mlsanity.com/paper/2603.19575/) (2026)

## Abstract (arXiv)

This work addresses the task of open world semantic segmentation using RGBD sensing to discover new semantic classes over time. Although there are many types of objects in the real-word, current semantic segmentation methods make a closed world assumption and are trained only to segment a limited number of object classes. Towards a more open world approach, we propose a novel method that incrementally learns new classes for image segmentation. The proposed system first segments each RGBD frame using both color and geometric information, and then aggregates that information to build a single segmented dense 3D map of the environment. The segmented 3D map representation is a key component of our approach as it is used to discover new object classes by identifying coherent regions in the 3D map that have no semantic label. The use of coherent region in the 3D map as a primitive element, rather than traditional elements such as surfels or voxels, also significantly reduces the computational complexity and memory use of our method. It thus leads to semi-real-time performance at {10.7}Hz when incrementally updating the dense 3D map at every frame. Through experiments on the NYUDv2 dataset, we demonstrate that the proposed method is able to correctly cluster objects of both known and unseen classes. We also show the quantitative comparison with the state-of-the-art supervised methods, the processing time of each step, and the influences of each component.