mlsanityData to 14 Oct 2026

Unseen Object Instance Segmentation for Robotic Environments

arXiv 2007.08073 ↗First posted 16 Jul 2020cs.CV cs.ROcited by 85

Not on the map. Why?

Abstract (arXiv)

In order to function in unstructured environments, robots need the ability to recognize unseen objects. We take a step in this direction by tackling the problem of segmenting unseen object instances in tabletop environments. However, the type of large-scale real-world dataset required for this task typically does not exist for most robotic settings, which motivates the use of synthetic data. Our proposed method, UOIS-Net, separately leverages synthetic RGB and synthetic depth for unseen object instance segmentation. UOIS-Net is comprised of two stages: first, it operates only on depth to produce object instance center votes in 2D or 3D and assembles them into rough initial masks. Secondly, these initial masks are refined using RGB. Surprisingly, our framework is able to learn from synthetic RGB-D data where the RGB is non-photorealistic. To train our method, we introduce a large-scale synthetic dataset of random objects on tabletops. We show that our method can produce sharp and accurate segmentation masks, outperforming state-of-the-art methods on unseen object instance segmentation. We also show that our method can segment unseen objects for robot grasping.

Key facts. Cited by 85 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 12; 2022: 16; 2023: 26; 2024: 15; 2025: 10; 2026: 6.

85 papers.

Sort:

By area:

  1. 2022MegaPose: 6D Pose Estimation of Novel Objects via Render & Comparecited by 201
  2. 2022Superpoint Transformer for 3D Scene Instance Segmentationcited by 116
  3. 2021Learning Open-World Object Proposals without Learning to Classifycited by 109
  4. 2023CNOS: A Strong Baseline for CAD-based Novel Object Segmentationcited by 87
  5. 2022SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fieldscited by 51
  6. 2021CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulationcited by 49
  7. 2021Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modelingcited by 43
  8. 2021Long-Horizon Manipulation of Unknown Objects via Task and Motion Planning with Estimated Affordancescited by 40
  9. 2022Affordance Learning from Play for Sample-Efficient Policy Learningcited by 30
  10. 2022Towards Scale Balanced 6-DoF Grasp Detection in Cluttered Scenescited by 21
  11. 2023ZeroPose: CAD-Prompted Zero-shot Object 6D Pose Estimation in Cluttered Scenescited by 20
  12. 2022Mean Shift Mask Transformer for Unseen Object Instance Segmentationcited by 19
38 more
  1. 2021ICM-3D: Instantiated Category Modeling for 3D Instance Segmentationcited by 18
  2. 2023ARMBench: An Object-centric Benchmark Dataset for Robotic Manipulationcited by 18
  3. 2024Adapting Pre-Trained Vision Models for Novel Instance Detection and Segmentationcited by 17
  4. 2021Unknown Object Segmentation from Stereo Imagescited by 16
  5. 2021RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networkscited by 15
  6. 2024Towards Open-World Grasping with Large Vision-Language Modelscited by 15
  7. 2023CenterGrasp: Object-Aware Implicit Representation Learning for Simultaneous Shape Reconstruction and 6-DoF Grasp Estimationcited by 13
  8. 2022NeuralBF: Neural Bilateral Filtering for Top-down Instance Segmentation on Point Cloudscited by 12
  9. 2023Self-Supervised Unseen Object Instance Segmentation via Long-Term Robot Interactioncited by 10
  10. 2025USIS16K: High-Quality Dataset for Underwater Salient Instance Segmentationcited by 10
  11. 2023Radar Instance Transformer: Reliable Moving Instance Segmentation in Sparse Radar Point Cloudscited by 9
  12. 2021Benchmarking the Robustness of Instance Segmentation Modelscited by 8
  13. 2022Unseen Object Instance Segmentation with Fully Test-time RGB-D Embeddings Adaptationcited by 8
  14. 2022Self-Supervised Interactive Object Segmentation Through a Singulation-and-Grasping Approachcited by 8
  15. 2023Open-world Instance Segmentation: Top-down Learning with Bottom-up Supervisioncited by 8
  16. 2025GraspClutter6D: A Large-scale Real-world Dataset for Robust Perception and Grasping in Cluttered Scenescited by 8
  17. 2023A Neuromorphic Dataset for Object Segmentation in Indoor Cluttered Environmentcited by 7
  18. 2023Sim-Suction: Learning a Suction Grasp Policy for Cluttered Environments Using a Synthetic Benchmarkcited by 7
  19. 2023SA6D: Self-Adaptive Few-Shot 6D Pose Estimator for Novel and Occluded Objectscited by 6
  20. 2021LRGNet: Learnable Region Growing for Class-Agnostic Point Cloud Segmentationcited by 5
  21. 2022Towards Robust Part-aware Instance Segmentation for Industrial Bin Pickingcited by 5
  22. 2022World Robot Challenge 2020 -- Partner Robot: A Data-Driven Approach for Room Tidying with Mobile Manipulatorcited by 5
  23. 2023Bimodal SegNet: Instance Segmentation Fusing Events and RGB Frames for Robotic Graspingcited by 5
  24. 2023Persistent Homology Meets Object Unity: Object Recognition in Cluttercited by 5
  25. 2024QueSTMaps: Queryable Semantic Topological Maps for 3D Scene Understandingcited by 5
  26. 2025ZISVFM: Zero-Shot Object Instance Segmentation in Indoor Robotic Environments with Vision Foundation Modelscited by 5
  27. 2021SIMstack: A Generative Shape and Instance Model for Unordered Object Stackscited by 4
  28. 2022Real-time Mapping of Physical Scene Properties with an Autonomous Robot Experimentercited by 4
  29. 2022SupeRGB-D: Zero-shot Instance Segmentation in Cluttered Indoor Environmentscited by 4
  30. 2023STOW: Discrete-Frame Segmentation and Tracking of Unseen Objects for Warehouse Picking Robotscited by 4
  31. 2024RISeg: Robot Interactive Object Segmentation via Body Frame-Invariant Featurescited by 4
  32. 2024Robot Instance Segmentation with Few Annotations for Graspingcited by 4
  33. 2024MSTA3D: Multi-scale Twin-attention for 3D Instance Segmentationcited by 4
  34. 2025Conditional Latent Diffusion Models for Zero-Shot Instance Segmentationcited by 4
  35. 2025SegDINO3D: 3D Instance Segmentation Empowered by Both Image-Level and Object-Level 2D Featurescited by 4
  36. 2021Depth-aware Object Segmentation and Grasp Detection for Robotic Picking Taskscited by 3
  37. 2022Learn Fast, Segment Well: Fast Object Segmentation Learning on the iCub Robotcited by 3
  38. 2023DoUnseen: Tuning-Free Class-Adaptive Object Detection of Unseen Objects for Robotic Graspingcited by 3

50 most cited of 85 All 85 (JSON)