# 3D Indoor Instance Segmentation in an Open-World

arXiv: https://arxiv.org/abs/2309.14338 · HTML: https://mlsanity.com/paper/2309.14338/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 42 references that are in the map.
- First posted on arXiv 25 Sep 2023.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [Visual Relationship Detection with Language Priors](https://mlsanity.com/paper/1608.00187/) (2016)
- [PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation](https://mlsanity.com/paper/2004.01658/) (2020)
- [Mask3D: Mask Transformer for 3D Semantic Instance Segmentation](https://mlsanity.com/paper/2210.03105/) (2022)
- [Towards Open World Object Detection](https://mlsanity.com/paper/2103.02603/) (2021)
- [Language-Grounded Indoor 3D Semantic Segmentation in the Wild](https://mlsanity.com/paper/2204.07761/) (2022)
- [Towards Open World Recognition](https://mlsanity.com/paper/1412.5687/) (2014)
- [3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans](https://mlsanity.com/paper/1812.07003/) (2018)
- [Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts](https://mlsanity.com/paper/2012.09165/) (2020)
- [Zero-Shot Object Detection](https://mlsanity.com/paper/1804.04340/) (2018)
- [OccuSeg: Occupancy-aware 3D Instance Segmentation](https://mlsanity.com/paper/2003.06537/) (2020)
- [SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation](https://mlsanity.com/paper/1711.08588/) (2017)
- [Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds](https://mlsanity.com/paper/1906.01140/) (2019)
- [GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud](https://mlsanity.com/paper/1812.03320/) (2018)
- [3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation](https://mlsanity.com/paper/2003.13867/) (2020)
- [Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentation](https://mlsanity.com/paper/2104.04691/) (2021)
- [OW-DETR: Open-world Detection Transformer](https://mlsanity.com/paper/2112.01513/) (2021)
- [Hierarchical Aggregation for 3D Instance Segmentation](https://mlsanity.com/paper/2108.02350/) (2021)
- [Superpoint Transformer for 3D Scene Instance Segmentation](https://mlsanity.com/paper/2211.15766/) (2022)
- [Dropout Sampling for Robust Object Detection in Open-Set Conditions](https://mlsanity.com/paper/1710.06677/) (2017)
- [Learning Open-World Object Proposals without Learning to Classify](https://mlsanity.com/paper/2108.06753/) (2021)
- [Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks](https://mlsanity.com/paper/2108.07478/) (2021)
- [Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Models](https://mlsanity.com/paper/2207.11514/) (2022)
- [CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D Dense CLIP](https://mlsanity.com/paper/2303.04748/) (2023)
- [3D Instance Segmentation via Multi-Task Metric Learning](https://mlsanity.com/paper/1906.08650/) (2019)

## Built on by (most cited first, 7 of 7)

- [Open-YOLO 3D: Towards Fast and Accurate Open-Vocabulary 3D Instance Segmentation](https://mlsanity.com/paper/2406.02548/) (2024)
- [Shape2Scene: 3D Scene Representation Learning Through Pre-training on Shape Data](https://mlsanity.com/paper/2407.10200/) (2024)
- [Towards 3D Objectness Learning in an Open World](https://mlsanity.com/paper/2510.17686/) (2025)
- [CLIMB-3D: Continual Learning for Imbalanced 3D Instance Segmentation](https://mlsanity.com/paper/2502.17429/) (2025)
- [A Novel Decomposed Feature-Oriented Framework for Open-Set Semantic Segmentation on LiDAR Data](https://mlsanity.com/paper/2503.11097/) (2025)
- [SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation](https://mlsanity.com/paper/2603.06572/) (2026)
- [ClickSeg3D: Few-Click Interactive Segmentation via Semantic Embeddings](https://mlsanity.com/paper/2605.08925/) (2026)

## Abstract (arXiv)

Existing 3D instance segmentation methods typically assume that all semantic classes to be segmented would be available during training and only seen categories are segmented at inference. We argue that such a closed-world assumption is restrictive and explore for the first time 3D indoor instance segmentation in an open-world setting, where the model is allowed to distinguish a set of known classes as well as identify an unknown object as unknown and then later incrementally learning the semantic category of the unknown when the corresponding category labels are available. To this end, we introduce an open-world 3D indoor instance segmentation method, where an auto-labeling scheme is employed to produce pseudo-labels during training and induce separation to separate known and unknown category labels. We further improve the pseudo-labels quality at inference by adjusting the unknown class probability based on the objectness score distribution. We also introduce carefully curated open-world splits leveraging realistic scenarios based on inherent object distribution, region-based indoor scene exploration and randomness aspect of open-world classes. Extensive experiments reveal the efficacy of the proposed contributions leading to promising open-world 3D instance segmentation performance.