mlsanityData to 7 Oct 2026

SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation

arXiv 2311.15707 ↗First posted 27 Nov 2023cs.CVcited by 115

Not on the map. Why?

Abstract (arXiv)

Zero-shot 6D object pose estimation involves the detection of novel objects with their 6D poses in cluttered scenes, presenting significant challenges for model generalizability. Fortunately, the recent Segment Anything Model (SAM) has showcased remarkable zero-shot transfer performance, which provides a promising solution to tackle this task. Motivated by this, we introduce SAM-6D, a novel framework designed to realize the task through two steps, including instance segmentation and pose estimation. Given the target objects, SAM-6D employs two dedicated sub-networks, namely Instance Segmentation Model (ISM) and Pose Estimation Model (PEM), to perform these steps on cluttered RGB-D images. ISM takes SAM as an advanced starting point to generate all possible object proposals and selectively preserves valid ones through meticulously crafted object matching scores in terms of semantics, appearance and geometry. By treating pose estimation as a partial-to-partial point matching problem, PEM performs a two-stage point matching process featuring a novel design of background tokens to construct dense 3D-3D correspondence, ultimately yielding the pose estimates. Without bells and whistles, SAM-6D outperforms the existing methods on the seven core datasets of the BOP Benchmark for both instance segmentation and pose estimation of novel objects.

Key facts. Cited by 115 papers in the map (data through 7 Oct 2026). Builds on 65 of its 73 references that are in the map. First posted on arXiv 27 Nov 2023.

Builds on

65 of 73 references.

References · 65

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2017Mask R-CNNcited by 7,259
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  8. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  9. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  10. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  11. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  12. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
38 more
  1. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 352
  2. 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 345
  3. 2023Segment Anything Model for Medical Image Analysis: an Experimental Studycited by 291
  4. 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
  5. 2020PREDATOR: Registration of 3D Point Clouds with Low Overlapcited by 287
  6. 2023Fast Segment Anythingcited by 286
  7. 2023Personalize Segment Anything Model with One Shotcited by 250
  8. 2023Inpaint Anything: Segment Anything Meets Image Inpaintingcited by 233
  9. 2023Track Anything: Segment Anything Meets Videoscited by 230
  10. 2019PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimationcited by 204
  11. 2022Geometric Transformer for Fast and Robust Point Cloud Registrationcited by 200
  12. 2022MegaPose: 6D Pose Estimation of Novel Objects via Render & Comparecited by 198
  13. 2023FLatten Transformer: Vision Transformer using Focused Linear Attentioncited by 170
  14. 2020Shape Prior Deformation for Categorical 6D Object Pose and Size Estimationcited by 158
  15. 2022OnePose: One-Shot Object Pose Estimation without CAD Modelscited by 155
  16. 2023SAM3D: Segment Anything in 3D Scenescited by 155
  17. 2021GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimatorcited by 148
  18. 2021FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimationcited by 144
  19. 2023Matcher: Segment Anything with One Shot Using All-Purpose Feature Matchingcited by 142
  20. 2023Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applicationscited by 121
  21. 2022Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Imagescited by 120
  22. 2022ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimationcited by 115
  23. 2023OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Modelscited by 114
  24. 2023GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondencecited by 113
  25. 2021DualPoseNet: Category-level 6D Object Pose and Size Estimation Using Dual Pose Network with Refined Learning of Pose Consistencycited by 103
  26. 2023Can SAM Segment Anything? When SAM Meets Camouflaged Object Detectioncited by 99
  27. 2023InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Languagecited by 96
  28. 2021SurfEmb: Dense and Continuous Correspondence Distributions for Object Pose Estimation with Learnt Surface Embeddingscited by 95
  29. 2023Anything-3D: Towards Single-view Anything Reconstruction in the Wildcited by 91
  30. 2023CNOS: A Strong Baseline for CAD-based Novel Object Segmentationcited by 86
  31. 2022FS6D: Few-Shot 6D Pose Estimation of Novel Objectscited by 81
  32. 2022OSOP: A Multi-Stage One Shot Object Pose Estimation Frameworkcited by 81
  33. 2023SAM Struggles in Concealed Scenes -- Empirical Study on Segment Anythingcited by 75
  34. 2022Category-Level 6D Object Pose and Size Estimation using Self-Supervised Deep Prior Deformation Networkscited by 73
  35. 2023BOP Challenge 2022 on Detection, Segmentation and Pose Estimation of Specific Rigid Objectscited by 70
  36. 2023InstructEdit: Improving Automatic Masks for Diffusion-based Image Editing With User Instructionscited by 64
  37. 2022Zero-Shot Category-Level Object Pose Estimationcited by 57
  38. 2022Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusionscited by 54

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 25; 2025: 57; 2026: 29.

115 papers.

Sort:

By area:

  1. 2023FoundPose: Unseen Object Pose Estimation with Foundation Featurescited by 109
  2. 2024SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Treecited by 77
  3. 2024BOP Challenge 2023 on Detection, Segmentation and Pose Estimation of Seen and Unseen Rigid Objectscited by 60
  4. 2023FreeZe: Training-free zero-shot 6D pose estimation with geometric and vision foundation modelscited by 56
  5. 2024SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priorscited by 53
  6. 2024Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Modelscited by 51
  7. 2025Any6D: Model-free 6D Pose Estimation of Novel Objectscited by 47
  8. 2023DeformGS: Scene Flow in Highly Deformable Scenes for Deformable Object Manipulationcited by 42
  9. 2024Deep Learning-Based Object Pose Estimation: A Comprehensive Surveycited by 39
  10. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 34
  11. 2023Improving the Generalization of Segmentation Foundation Model under Distribution Shift via Weakly Supervised Adaptationcited by 31
  12. 2025HUMOTO: A 4D Dataset of Mocap Human Object Interactionscited by 30
38 more
  1. 2024F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactionscited by 24
  2. 2025BOP Challenge 2024 on Model-Based and Model-Free 6D Object Pose Estimationcited by 18
  3. 2024Adapting Pre-Trained Vision Models for Novel Instance Detection and Segmentationcited by 17
  4. 2025One2Any: One-Reference 6D Pose Estimation for Any Objectcited by 17
  5. 2025GCE-Pose: Global Context Enhancement for Category-level Object Pose Estimationcited by 16
  6. 2024GS-Pose: Generalizable Segmentation-based 6D Object Pose Estimation with 3D Gaussian Splattingcited by 14
  7. 2025Co-op: Correspondence-based Novel Object Pose Estimationcited by 13
  8. 2025Accurate and efficient zero-shot 6D pose estimation with frozen foundation modelscited by 13
  9. 2024Segment Anything with Multiple Modalitiescited by 12
  10. 2024UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Imagecited by 12
  11. 2024IKEA Manuals at Work: 4D Grounding of Assembly Instructions on Internet Videoscited by 11
  12. 2025Scalable Unseen Objects 6-DoF Absolute Pose Estimation with Robotic Integrationcited by 11
  13. 2024High-resolution open-vocabulary object 6D pose estimationcited by 10
  14. 20246DOPE-GS: Online 6D Object Pose Estimation using Gaussian Splattingcited by 10
  15. 2025UA-Pose: Uncertainty-Aware 6D Object Pose Estimation and Online Object Completion with Partial Referencescited by 9
  16. 2025XYZ-IBD: Benchmarking Robust 6D Object Pose Estimation under Real-World Industrial Complexitycited by 7
  17. 2024Manual-PA: Learning 3D Part Assembly from Instruction Diagramscited by 6
  18. 2025RA-DP: Rapid Adaptive Diffusion Policy for Training-Free High-frequency Robotics Replanningcited by 6
  19. 2025RGBTrack: Fast, Robust Depth-Free 6D Pose Estimation and Trackingcited by 6
  20. 2025GelSLAM: A Real-time, High-Fidelity, and Robust 3D Tactile SLAM Systemcited by 6
  21. 2025Uncertainty-Aware Knowledge Distillation for Compact and Efficient 6DoF Pose Estimationcited by 5
  22. 2025PicoPose: Progressive Pixel-to-Pixel Correspondence Learning for Novel Object Pose Estimationcited by 5
  23. 2025ConceptPose: Training-Free Zero-Shot Object Pose Estimation using Concept Vectorscited by 5
  24. 2024Diffusion Features for Zero-Shot 6DoF Object Pose Estimationcited by 4
  25. 2025HIPPo: Harnessing Image-to-3D Priors for Model-free Zero-shot 6D Pose Estimationcited by 4
  26. 2025V-HOP: Visuo-Haptic 6D Object Pose Trackingcited by 4
  27. 2025Distilling 3D distinctive local descriptors for 6D pose estimationcited by 4
  28. 2025Deep Depth Estimation from Thermal Image: Dataset, Benchmark, and Challengescited by 4
  29. 2025Conditional Latent Diffusion Models for Zero-Shot Instance Segmentationcited by 4
  30. 2025RayPose: Ray Bundling Diffusion for Template Views in Unseen 6D Object Pose Estimationcited by 4
  31. 2024Learning Embeddings with Centroid Triplet Loss for Object Identification in Robotic Graspingcited by 3
  32. 2024Comparative Evaluation of 3D Reconstruction Methods for Object Pose Estimationcited by 3
  33. 2024BOP-Distrib: Revisiting 6D Pose Estimation Benchmarks for Better Evaluation under Visual Ambiguitiescited by 3
  34. 2024GFreeDet: Exploiting Gaussian Splatting and Foundation Models for Model-free Unseen Object Detection in the BOP Challenge 2024cited by 3
  35. 2025ViTa-Zero: Zero-shot Visuotactile Object 6D Pose Estimationcited by 3
  36. 2025SenseShift6D: Multimodal RGB-D Benchmarking for Robust 6D Pose Estimation across Environment and Sensor Variationscited by 3
  37. 2025MUSE: Model-based Uncertainty-aware Similarity Estimation for zero-shot 2D Object Detection and Segmentationcited by 3
  38. 2026Text-Guided 6D Object Pose Rearrangement via Closed-Loop VLM Agentscited by 3

50 most cited of 115 All 115 (JSON)