mlsanityData to 13 Oct 2026

GrabS: Generative Embodied Agent for 3D Object Segmentation without Scene Supervision

arXiv 2504.11754 ↗First posted 16 Apr 2025cs.CV cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

We study the hard problem of 3D object segmentation in complex point clouds without requiring human labels of 3D scenes for supervision. By relying on the similarity of pretrained 2D features or external signals such as motion to group 3D points as objects, existing unsupervised methods are usually limited to identifying simple objects like cars or their segmented objects are often inferior due to the lack of objectness in pretrained features. In this paper, we propose a new two-stage pipeline called GrabS. The core concept of our method is to learn generative and discriminative object-centric priors as a foundation from object datasets in the first stage, and then design an embodied agent to learn to discover multiple objects by querying against the pretrained generative priors in the second stage. We extensively evaluate our method on two real-world datasets and a newly created synthetic dataset, demonstrating remarkable segmentation performance, clearly surpassing all existing unsupervised methods.

Key facts. Cited by 3 papers in the map (data through 13 Oct 2026). Builds on 58 of its 69 references that are in the map. First posted on arXiv 16 Apr 2025.

Builds on

58 of 69 references.

References · 58

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,952
  3. 2017Proximal Policy Optimization Algorithmscited by 14,592
  4. 2023Segment Anythingcited by 8,195
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,857
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,915
  7. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,389
  8. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,096
  9. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,889
  10. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,686
  11. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,205
  12. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 976
38 more
  1. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 747
  2. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 734
  3. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
  4. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 503
  5. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 406
  6. 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 384
  7. 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 300
  8. 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 288
  9. 2020Neural Unsigned Distance Fields for Implicit Function Learningcited by 265
  10. 20183D-SIS: 3D Semantic Instance Segmentation of RGB-D Scanscited by 217
  11. 2020Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contextscited by 210
  12. 2022SoftGroup for 3D Instance Segmentation on Point Cloudscited by 197
  13. 2020OccuSeg: Occupancy-aware 3D Instance Segmentationcited by 177
  14. 2017SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentationcited by 169
  15. 2019Learning Object Bounding Boxes for 3D Instance Segmentation on Point Cloudscited by 156
  16. 2018GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloudcited by 146
  17. 2022Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functionscited by 133
  18. 2022Diffusion-SDF: Text-to-Shape via Voxelized Diffusioncited by 128
  19. 2021Hierarchical Aggregation for 3D Instance Segmentationcited by 127
  20. 2023Segment Any Point Cloud Sequences by Distilling Vision Foundation Modelscited by 123
  21. 2023Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidancecited by 123
  22. 2022Superpoint Transformer for 3D Scene Instance Segmentationcited by 115
  23. 2017Accelerated Hierarchical Density Clusteringcited by 101
  24. 2020SoftFlow: Probabilistic Framework for Normalizing Flow on Manifoldscited by 98
  25. 2022Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Modelscited by 95
  26. 2023OneFormer3D: One Transformer for Unified Point Cloud Segmentationcited by 95
  27. 2023SAI3D: Segment Any Instance in 3D Scenescited by 90
  28. 2020Discrete Point Flow Networks for Efficient Point Cloud Generationcited by 87
  29. 2023OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentationcited by 82
  30. 2023Open-Vocabulary Point-Cloud Object Detection without 3D Annotationcited by 81
  31. 2021SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Datacited by 78
  32. 2020DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolutioncited by 74
  33. 2024MaskClustering: View Consensus based Mask Graph Clustering for Open-Vocabulary 3D Instance Segmentationcited by 56
  34. 2023Mask-Attention-Free Transformer for 3D Instance Segmentationcited by 45
  35. 2023UnScene3D: Unsupervised 3D Instance Segmentation for Indoor Scenescited by 31
  36. 2024Open-YOLO 3D: Towards Fast and Accurate Open-Vocabulary 3D Instance Segmentationcited by 30
  37. 2020GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-aware Supervisioncited by 28
  38. 2022OGC: Unsupervised 3D Object Segmentation from Rigid Dynamics of Point Cloudscited by 24

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026EvObj: Learning Evolving Object-centric Representations for 3D Instance Segmentation without Scene Supervisioncited by 1
  2. 20263D-DLP: Self-Supervised 3D Object-Centric Scene Representation Learningcited by 1
  3. 2026FoundObj: Self-supervised Foundation Models as Rewards for Label-free 3D Object Segmentationcited by 0