mlsanityData to 7 Oct 2026

ZS6D: Zero-shot 6D Object Pose Estimation using Vision Transformers

arXiv 2309.11986 ↗First posted 21 Sep 2023cs.CVcited by 45

Not on the map. Why?

Abstract (arXiv)

As robotic systems increasingly encounter complex and unconstrained real-world scenarios, there is a demand to recognize diverse objects. The state-of-the-art 6D object pose estimation methods rely on object-specific training and therefore do not generalize to unseen objects. Recent novel object pose estimation methods are solving this issue using task-specific fine-tuned CNNs for deep template matching. This adaptation for pose estimation still requires expensive data rendering and training procedures. MegaPose for example is trained on a dataset consisting of two million images showing 20,000 different objects to reach such generalization capabilities. To overcome this shortcoming we introduce ZS6D, for zero-shot novel object 6D pose estimation. Visual descriptors, extracted using pre-trained Vision Transformers (ViT), are used for matching rendered templates against query images of objects and for establishing local correspondences. These local correspondences enable deriving geometric correspondences and are used for estimating the object's 6D pose with RANSAC-based PnP. This approach showcases that the image descriptors extracted by pre-trained ViTs are well-suited to achieve a notable improvement over two state-of-the-art novel object 6D pose estimation methods, without the need for task-specific fine-tuning. Experiments are performed on LMO, YCBV, and TLESS. In comparison to one of the two methods we improve the Average Recall on all three datasets and compared to the second method we improve on two datasets.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 33 of its 41 references that are in the map. First posted on arXiv 21 Sep 2023.

Builds on

33 of 41 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2023Segment Anythingcited by 8,158
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2017Mask R-CNNcited by 7,259
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  9. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  10. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  11. 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
  12. 2018DeepIM: Deep Iterative Matching for 6D Pose Estimationcited by 283
21 more
  1. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
  2. 2019Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimationcited by 218
  3. 2017T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objectscited by 210
  4. 2022MegaPose: 6D Pose Estimation of Novel Objects via Render & Comparecited by 198
  5. 2019DPOD: 6D Pose Object Detector and Refinercited by 189
  6. 2021GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimatorcited by 148
  7. 2020EPOS: Estimating 6D Pose of Objects with Symmetriescited by 147
  8. 2022ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimationcited by 115
  9. 2023CNOS: A Strong Baseline for CAD-based Novel Object Segmentationcited by 86
  10. 2022OSOP: A Multi-Stage One Shot Object Pose Estimation Frameworkcited by 81
  11. 2022Zero-Shot Category-Level Object Pose Estimationcited by 57
  12. 2022Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusionscited by 54
  13. 2019Multi-path Learning for Object Pose Estimation Across Domainscited by 52
  14. 2021Single-Stage Keypoint-Based Category-Level Object Pose Estimation from an RGB Imagecited by 42
  15. 2023POPE: 6-DoF Promptable Pose Estimation of Any Object, in Any Scene, with One Referencecited by 30
  16. 2022Towards Bridging the Space Domain Gap for Satellite Pose Estimation using Event Sensingcited by 21
  17. 2020PyraPose: Feature Pyramids for Fast and Accurate Object Pose Estimation under Domain Shiftcited by 14
  18. 2022COPE: End-to-end trainable Constant Runtime Object Pose Estimationcited by 10
  19. 2021KDFNet: Learning Keypoint Distance Field for 6D Object Pose Estimationcited by 9
  20. 2023Self-supervised Vision Transformers for 3D Pose Estimation of Novel Objectscited by 9
  21. 2020Handling Object Symmetries in CNN-based Pose Estimationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 25; 2026: 12.

45 papers.

Sort:

By area:

  1. 2023GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondencecited by 113
  2. 2023FoundPose: Unseen Object Pose Estimation with Foundation Featurescited by 109
  3. 2023FreeZe: Training-free zero-shot 6D pose estimation with geometric and vision foundation modelscited by 56
  4. 2024Deep Learning-Based Object Pose Estimation: A Comprehensive Surveycited by 39
  5. 2025Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrationscited by 33
  6. 2025One2Any: One-Reference 6D Pose Estimation for Any Objectcited by 17
  7. 2025GCE-Pose: Global Context Enhancement for Category-level Object Pose Estimationcited by 16
  8. 2025Co-op: Correspondence-based Novel Object Pose Estimationcited by 13
  9. 2025Accurate and efficient zero-shot 6D pose estimation with frozen foundation modelscited by 13
  10. 2024Zero123-6D: Zero-shot Novel View Synthesis for RGB Category-level 6D Pose Estimationcited by 10
  11. 2025GoTrack: Generic 6DoF Object Pose Refinement and Trackingcited by 10
  12. 2025UA-Pose: Uncertainty-Aware 6D Object Pose Estimation and Online Object Completion with Partial Referencescited by 9
33 more
  1. 2024Out-of-Distribution Recovery with Object-Centric Keypoint Inverse Policy for Visuomotor Imitation Learningcited by 6
  2. 2025RGBTrack: Fast, Robust Depth-Free 6D Pose Estimation and Trackingcited by 6
  3. 2024Diffusion Features for Zero-Shot 6DoF Object Pose Estimationcited by 4
  4. 2025HIPPo: Harnessing Image-to-3D Priors for Model-free Zero-shot 6D Pose Estimationcited by 4
  5. 2025Distilling 3D distinctive local descriptors for 6D pose estimationcited by 4
  6. 2025RayPose: Ray Bundling Diffusion for Template Views in Unseen 6D Object Pose Estimationcited by 4
  7. 2025ZeroBP: Learning Position-Aware Correspondence for Zero-shot 6D Pose Estimation in Bin-Pickingcited by 2
  8. 2025Reasoning and Learning a Perceptual Metric for Self-Training of Reflective Objects in Bin-Picking with a Low-cost Cameracited by 2
  9. 2024Generalizable Single-view Object Pose Estimation by Two-side Generating and Matchingcited by 1
  10. 2025Multi-Modal 3D Mesh Reconstruction from Images and Textcited by 1
  11. 2025Foundation Feature-Driven Online End-Effector Pose Estimation: A Marker-Free and Learning-Free Approachcited by 1
  12. 2025RefPose: Leveraging Reference Geometric Correspondences for Accurate 6D Pose Estimation of Unseen Objectscited by 1
  13. 2025Zero-shot Inexact CAD Model Alignment from a Single Imagecited by 1
  14. 2025PASG: A Closed-Loop Framework for Automated Geometric Primitive Extraction and Semantic Anchoring in Robotic Manipulationcited by 1
  15. 2026IndustryShapes: An RGB-D Benchmark dataset for 6D object pose estimation of industrial assembly components and toolscited by 1
  16. 2025PoseIDON: 6DoF Pose Estimation with Foundation Model Features for Marine Sediment Burial Mappingcited by 0
  17. 2025RoboEye: Enhancing 2D Robotic Object Identification with Selective 3D Geometric Keypoint Matchingcited by 0
  18. 2025Color-Pair Guided Robust Zero-Shot 6D Pose Estimation and Tracking of Cluttered Objects on Edge Devicescited by 0
  19. 2025SCOPE: Semantic Conditioning for Sim2Real Category-Level Object Pose Estimation in Roboticscited by 0
  20. 2025OPFormer: Object Pose Estimation leveraging foundation model with geometric encodingcited by 0
  21. 2025Geo6DPose: Fast Zero-Shot 6D Object Pose Estimation via Geometry-Filtered Feature Matchingcited by 0
  22. 2025PoseGAM: Robust Unseen Object Pose Estimation via Geometry-Aware Multi-View Reasoningcited by 0
  23. 2026MixRI: Mixing Features of Reference Images for Novel Object Pose Estimationcited by 0
  24. 2026Detecting 3D Line Segments for 6DoF Pose Estimation with Limited Datacited by 0
  25. 2026Finding NeMO: A Geometry-Aware Representation of Template Views for Few-Shot Perceptioncited by 0
  26. 2026MooMIns -- Monocular 3D Reconstruction and Object Pose Estimation from Multiple Instancescited by 0
  27. 2026Leveraging Deep Learning for Object and Position Recognition of Load Carriers for Autonomous Logistics Vehiclescited by 0
  28. 2026Pose Anything Anywhere:Model-free Object Poses from Arbitrary Referencescited by 0
  29. 2026MF-UAVPose6D: A Model-Free Monocular 6-DoF Pose Estimation Framework for Fixed-Wing UAVscited by 0
  30. 2026PIXIE: A Zero-Shot texture-invariant 6D pose estimation framework for unseen objects with assembly defectscited by 0
  31. 2026Robust PnP on a Neuromorphic Processor for Object Pose Estimationcited by 0
  32. 2026RRTrack: Robust and Recoverable Object 6D Pose Tracking for Dynamic Scenescited by 0
  33. 2026Back to the Feature: Zero-Shot 6DoF Pose Estimation via Dense Local Featurescited by 0