mlsanityData to 14 Oct 2026

SAM3D: Segment Anything in 3D Scenes

arXiv 2306.03908 ↗First posted 6 Jun 2023cs.CVcited by 157

Not on the map. Why?

Abstract (arXiv)

In this work, we propose SAM3D, a novel framework that is able to predict masks in 3D point clouds by leveraging the Segment-Anything Model (SAM) in RGB images without further training or finetuning. For a point cloud of a 3D scene with posed RGB images, we first predict segmentation masks of RGB images with SAM, and then project the 2D masks into the 3D points. Later, we merge the 3D masks iteratively with a bottom-up merging approach. At each step, we merge the point cloud masks of two adjacent frames with the bidirectional merging approach. In this way, the 3D masks predicted from different frames are gradually merged into the 3D masks of the whole 3D scene. Finally, we can optionally ensemble the result from our SAM3D with the over-segmentation results based on the geometric information of the 3D scenes. Our approach is experimented with ScanNet dataset and qualitative results demonstrate that our SAM3D achieves reasonable and fine-grained 3D segmentation results without any training or finetuning of SAM.

Key facts. Cited by 157 papers in the map (data through 14 Oct 2026). Builds on 14 of its 16 references that are in the map. First posted on arXiv 6 Jun 2023.

Builds on

14 of 16 references.

References · 14

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2023Segment Anythingcited by 8,234
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  5. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  7. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 699
  8. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 657
  9. 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 388
  10. 20183DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentationcited by 118
  11. 2020Virtual Multi-view Fusion for 3D Semantic Segmentationcited by 99
  12. 2023Anything-3D: Towards Single-view Anything Reconstruction in the Wildcited by 92
2 more
  1. 2019Multi-view PointNet for 3D Scene Understandingcited by 76
  2. 2023SAD: Segment Any RGBDcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 11; 2024: 44; 2025: 57; 2026: 45.

157 papers.

Sort:

By area:

  1. 2023Gaussian Grouping: Segment and Edit Anything in 3D Scenescited by 393
  2. 2023Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidancecited by 124
  3. 2023SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimationcited by 117
  4. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 109
  5. 2024SAMPart3D: Segment Any Part in 3D Objectscited by 97
  6. 2025PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyondcited by 95
  7. 2023SAI3D: Segment Any Instance in 3D Scenescited by 90
  8. 2023OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentationcited by 82
  9. 2023Segment3D: Learning Fine-Grained Class-Agnostic 3D Segmentation without Manual Labelscited by 64
  10. 2025OmniPart: Part-Aware 3D Generation with Semantic Decoupling and Structural Cohesioncited by 61
  11. 2024MaskClustering: View Consensus based Mask Graph Clustering for Open-Vocabulary 3D Instance Segmentationcited by 56
  12. 2024Part123: Part-aware 3D Reconstruction from a Single-view Imagecited by 55
38 more
  1. 2024Point-SAM: Promptable 3D Segmentation Model for Point Cloudscited by 50
  2. 2023SAMPro3D: Locating SAM Prompts in 3D for Zero-Shot Instance Segmentationcited by 44
  3. 2024OA-CNNs: Omni-Adaptive Sparse CNNs for 3D Semantic Segmentationcited by 44
  4. 2025P3-SAM: Native 3D Part Segmentationcited by 43
  5. 2024EmbodiedSAM: Online Segment Any 3D Thing in Real Timecited by 40
  6. 2023SAM-guided Graph Cut for 3D Instance Segmentationcited by 35
  7. 2024Search3D: Hierarchical Open-Vocabulary 3D Segmentationcited by 31
  8. 2024GroupContrast: Semantic-aware Self-supervised Representation Learning for 3D Understandingcited by 26
  9. 2025ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splattingcited by 25
  10. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24
  11. 2023SlimSAM: 0.1% Data Makes Segment Anything Slimcited by 20
  12. 2025OpenSplat3D: Open-Vocabulary 3D Instance Segmentation using Gaussian Splattingcited by 20
  13. 2023ZeroPS: High-quality Cross-modal Knowledge Transfer for Zero-Shot 3D Part Segmentationcited by 18
  14. 2024PartSTAD: 2D-to-3D Part Segmentation Task Adaptationcited by 18
  15. 2024SAM2Point: Segment Any 3D as Videos in Zero-shot and Promptable Mannerscited by 18
  16. 2024Visual Foundation Models Boost Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentationcited by 15
  17. 2024A Unified Framework for 3D Scene Understandingcited by 15
  18. 2024Any3DIS: Class-Agnostic 3D Instance Segmentation by 2D Mask Trackingcited by 15
  19. 2024iSeg: Interactive 3D Segmentation via Interactive Attentioncited by 14
  20. 2024ASAM: Boosting Segment Anything Model with Adversarial Tuningcited by 12
  21. 2024Part2Object: Hierarchical Unsupervised 3D Instance Segmentationcited by 12
  22. 2025OnlineAnySeg: Online Zero-Shot 3D Segmentation by Visual Foundation Model Guided 2D Mask Mergingcited by 12
  23. 2024Generalizing 6-DoF Grasp Detection via Domain Prior Knowledgecited by 11
  24. 2024GoodSAM++: Bridging Domain and Capacity Gaps via Segment Anything Model for Panoramic Semantic Segmentationcited by 11
  25. 2025InstaScene: Towards Complete 3D Instance Decomposition and Reconstruction from Cluttered Scenescited by 11
  26. 20243x2: 3D Object Part Segmentation by 2D Semantic Correspondencescited by 10
  27. 2025Training-Free Hierarchical Scene Understanding for Gaussian Splatting with Superpoint Graphscited by 10
  28. 2025Masked Point-Entity Contrast for Open-Vocabulary 3D Scene Understandingcited by 10
  29. 2025Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perceptioncited by 10
  30. 2026SegviGen: Repurposing 3D Generative Model for Part Segmentationcited by 10
  31. 2024MixSup: Mixed-grained Supervision for Label-efficient LiDAR-based 3D Object Detectioncited by 9
  32. 2024PointSeg: A Training-Free Paradigm for 3D Scene Segmentation via Foundation Modelscited by 9
  33. 2024O2V-Mapping: Online Open-Vocabulary Mapping with Neural Implicit Representationcited by 9
  34. 2025PartUV: Part-Based UV Unwrapping of 3D Meshescited by 9
  35. 2025Details Matter for Indoor Open-vocabulary 3D Instance Segmentationcited by 8
  36. 2025MeshMosaic: Scaling Artist Mesh Generation via Local-to-Global Assemblycited by 8
  37. 2025Inpaint360GS: Efficient Object-Aware 3D Inpainting via Gaussian Splatting for 360deg Scenescited by 8
  38. 2025Unified Semantic Transformer for 3D Scene Understandingcited by 8

50 most cited of 157 All 157 (JSON)