mlsanityData to 7 Oct 2026

SAM2Point: Segment Any 3D as Videos in Zero-shot and Promptable Manners

arXiv 2408.16768 ↗First posted 29 Aug 2024cs.CV cs.AI cs.CLcited by 18

Not on the map. Why?

Abstract (arXiv)

We introduce SAM2Point, a preliminary exploration adapting Segment Anything Model 2 (SAM 2) for zero-shot and promptable 3D segmentation. SAM2Point interprets any 3D data as a series of multi-directional videos, and leverages SAM 2 for 3D-space segmentation, without further training or 2D-3D projection. Our framework supports various prompt types, including 3D points, boxes, and masks, and can generalize across diverse scenarios, such as 3D objects, indoor scenes, outdoor environments, and raw sparse LiDAR. Demonstrations on multiple 3D datasets, e.g., Objaverse, S3DIS, ScanNet, Semantic3D, and KITTI, highlight the robust generalization capabilities of SAM2Point. To our best knowledge, we present the most faithful implementation of SAM in 3D, which may serve as a starting point for future research in promptable 3D segmentation. Online Demo: https://huggingface.co/spaces/ZiyuG/SAM2Point . Code: https://github.com/ZiyuGuo99/SAM2Point .

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 36 of its 45 references that are in the map. First posted on arXiv 29 Aug 2024.

Builds on

36 of 45 references.

References · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Segment Anythingcited by 8,158
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  7. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  8. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  9. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  10. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  11. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  12. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
24 more
  1. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
  2. 2023Segment Anything Model for Medical Image Analysis: an Experimental Studycited by 291
  3. 2023Personalize Segment Anything Model with One Shotcited by 250
  4. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 231
  5. 2023Track Anything: Segment Anything Meets Videoscited by 230
  6. 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 199
  7. 2023Segment and Track Anythingcited by 193
  8. 2023Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Followingcited by 192
  9. 2022PointCLIP V2: Prompting CLIP and GPT for Powerful 3D Open-world Learningcited by 186
  10. 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 183
  11. 2023Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretrainingcited by 162
  12. 2022CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-trainingcited by 159
  13. 2017Semantic3D.net: A new Large-scale Point Cloud Classification Benchmarkcited by 158
  14. 2023SAM3D: Segment Anything in 3D Scenescited by 155
  15. 2022Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoderscited by 143
  16. 2023Matcher: Segment Anything with One Shot Using All-Purpose Feature Matchingcited by 142
  17. 2024SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Modelscited by 130
  18. 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenescited by 125
  19. 2023SAI3D: Segment Any Instance in 3D Scenescited by 90
  20. 2023Joint-MAE: 2D-3D Joint Masked Autoencoders for 3D Point Cloud Pre-trainingcited by 52
  21. 2024Point-SAM: Promptable 3D Segmentation Model for Point Cloudscited by 49
  22. 2023SAMPro3D: Locating SAM Prompts in 3D for Zero-Shot Instance Segmentationcited by 44
  23. 2023PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View Instance Segmentation and Maximum Likelihood Estimationcited by 37
  24. 2024Any2Point: Empowering Any-modality Large Models for Efficient 3D Understandingcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 11; 2026: 1.

18 papers.

Sort:

By area:

  1. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
  2. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 130
  3. 2025MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiencycited by 123
  4. 2024MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Enginescited by 98
  5. 2025PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyondcited by 91
  6. 2024PhysFlow: Unleashing the Potential of Multi-modal Foundation Models and Video Diffusion for 4D Dynamic Physical Scene Simulationcited by 36
  7. 2025Detect Anything 3D in the Wildcited by 32
  8. 2024PixWizard: Versatile Image-to-Image Visual Assistant with Open-Language Instructionscited by 31
  9. 2025LoHoVLA: A Unified Vision-Language-Action Model for Long-Horizon Embodied Taskscited by 25
  10. 2025SciVerse: Unveiling the Knowledge Comprehension and Visual Reasoning of LMMs on Multi-modal Scientific Problemscited by 22
  11. 2025Vanish into Thin Air: Cross-prompt Universal Adversarial Attacks for SAM2cited by 11
  12. 2025PiSA: A Self-Augmented Data Engine and Training Strategy for 3D Understanding with Large Modelscited by 8
6 more
  1. 2024CellSeg1: Robust Cell Segmentation with One Training Imagecited by 6
  2. 2025SALT: A Flexible Semi-Automatic Labeling Tool for General LiDAR Point Clouds with Cross-Scene Adaptability and 4D Consistencycited by 4
  3. 2025Synergistic Bleeding Region and Point Detection in Laparoscopic Surgical Videoscited by 3
  4. 2024Zero-Shot Pupil Segmentation with SAM 2: A Case Study of Over 14 Million Imagescited by 2
  5. 2024Test-time Correction: An Online 3D Detection System via Visual Promptingcited by 0
  6. 2026ClickSeg3D: Few-Click Interactive Segmentation via Semantic Embeddingscited by 0