mlsanityData to 7 Oct 2026

SupeRGB-D: Zero-shot Instance Segmentation in Cluttered Indoor Environments

arXiv 2212.11922 ↗First posted 22 Dec 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Object instance segmentation is a key challenge for indoor robots navigating cluttered environments with many small objects. Limitations in 3D sensing capabilities often make it difficult to detect every possible object. While deep learning approaches may be effective for this problem, manually annotating 3D data for supervised learning is time-consuming. In this work, we explore zero-shot instance segmentation (ZSIS) from RGB-D data to identify unseen objects in a semantic category-agnostic manner. We introduce a zero-shot split for Tabletop Objects Dataset (TOD-Z) to enable this study and present a method that uses annotated objects to learn the ``objectness'' of pixels and generalize to unseen object categories in cluttered indoor environments. Our method, SupeRGB-D, groups pixels into small patches based on geometric cues and learns to merge the patches in a deep agglomerative clustering fashion. SupeRGB-D outperforms existing baselines on unseen objects while achieving similar performance on seen objects. We further show competitive results on the real dataset OCID. With its lightweight design (0.4 MB memory requirement), our method is extremely suitable for mobile and robotic applications. Additional DINO features can increase performance with a higher memory requirement. The dataset split and code are available at https://github.com/evinpinar/supergb-d.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 33 of its 52 references that are in the map. First posted on arXiv 22 Dec 2022.

Builds on

33 of 52 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Mask R-CNNcited by 7,259
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  6. 2019Point-Voxel CNN for Efficient 3D Deep Learningcited by 343
  7. 2019Zero-Shot Semantic Segmentationcited by 245
  8. 20183D-SIS: 3D Semantic Instance Segmentation of RGB-D Scanscited by 216
  9. 2018Zero-Shot Object Detectioncited by 189
  10. 2017SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentationcited by 168
  11. 2022Multimodal Token Fusion for Vision Transformerscited by 132
  12. 2021Learning Open-World Object Proposals without Learning to Classifycited by 109
21 more
  1. 2020A Survey on Instance Segmentation: State of the artcited by 85
  2. 2020Unseen Object Instance Segmentation for Robotic Environmentscited by 85
  3. 2015Learning to Segment Object Candidatescited by 79
  4. 2020Few-shot 3D Point Cloud Semantic Segmentationcited by 77
  5. 2020Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentationcited by 64
  6. 2020Generalized Few-shot Semantic Segmentationcited by 59
  7. 2019EasyLabel: A Semi-Automatic Pixel-wise Object Annotation Tool for Creating Robotic RGB-D Datasetscited by 58
  8. 2021Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Cloudscited by 55
  9. 2021Incremental Few-Shot Instance Segmentationcited by 49
  10. 2021Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modelingcited by 43
  11. 2022Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinitycited by 41
  12. 2021Zero-Shot Instance Segmentationcited by 40
  13. 2021Novel Class Discovery in Semantic Segmentationcited by 31
  14. 2020Learning to Group: A Bottom-Up Framework for 3D Part Discovery in Unseen Categoriescited by 29
  15. 2022From 2D to 3D: Re-thinking Benchmarking of Monocular Depth Predictioncited by 19
  16. 2021SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereocited by 18
  17. 2021Unknown Object Segmentation from Stereo Imagescited by 16
  18. 20213D Compositional Zero-shot Learning with DeCompositional Consensuscited by 16
  19. 2021RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networkscited by 15
  20. 2020Segmenting Unseen Industrial Components in a Heavy Clutter Using RGB-D Fusion and Synthetic Datacited by 6
  21. 2021Object-aware Monocular Depth Prediction with Instance Convolutionscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 0; 2026: 1.

4 papers.

  1. 2023FoundPose: Unseen Object Pose Estimation with Foundation Featurescited by 109
  2. 2023ZeroPose: CAD-Prompted Zero-shot Object 6D Pose Estimation in Cluttered Scenescited by 20
  3. 2024Configurable Embodied Data Generation for Class-Agnostic RGB-D Video Segmentationcited by 0
  4. 2026Clutt3R-Seg: Sparse-view 3D Instance Segmentation for Language-grounded Grasping in Cluttered Scenescited by 0