mlsanityData to 14 Oct 2026

Open-vocabulary object 6D pose estimation

arXiv 2312.00690 ↗First posted 1 Dec 2023cs.CVcited by 29

Not on the map. Why?

Abstract (arXiv)

We introduce the new setting of open-vocabulary object 6D pose estimation, in which a textual prompt is used to specify the object of interest. In contrast to existing approaches, in our setting (i) the object of interest is specified solely through the textual prompt, (ii) no object model (e.g., CAD or video sequence) is required at inference, and (iii) the object is imaged from two RGBD viewpoints of different scenes. To operate in this setting, we introduce a novel approach that leverages a Vision-Language Model to segment the object of interest from the scenes and to estimate its relative 6D pose. The key of our approach is a carefully devised strategy to fuse object-level information provided by the prompt with local image features, resulting in a feature space that can generalize to novel concepts. We validate our approach on a new benchmark based on two popular datasets, REAL275 and Toyota-Light, which collectively encompass 34 object instances appearing in four thousand image pairs. The results demonstrate that our approach outperforms both a well-established hand-crafted method and a recent deep learning-based baseline in estimating the relative 6D pose of objects in different scenes. Code and dataset are available at https://jcorsetti.github.io/oryon.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 40 of its 59 references that are in the map. First posted on arXiv 1 Dec 2023.

Builds on

40 of 59 references.

References · 40

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  5. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  6. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  7. 2017Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentationscited by 620
  8. 2021Extract Free Dense Labels from CLIPcited by 583
  9. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 565
  10. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 495
  11. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
  12. 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 219
28 more
  1. 2022MegaPose: 6D Pose Estimation of Novel Objects via Render & Comparecited by 201
  2. 2020BOP Challenge 2020 on 6D Object Localizationcited by 193
  3. 2022OnePose: One-Shot Object Pose Estimation without CAD Modelscited by 157
  4. 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 148
  5. 2023PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustmentcited by 147
  6. 2021FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimationcited by 144
  7. 2020Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wildcited by 142
  8. 2021PointDSC: Robust Point Cloud Registration using Deep Spatial Consistencycited by 142
  9. 2018ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shapecited by 123
  10. 2022Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Imagescited by 121
  11. 2022ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimationcited by 116
  12. 2023OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Modelscited by 115
  13. 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wildcited by 102
  14. 2022REGTR: End-to-end Point Cloud Correspondences with Transformerscited by 96
  15. 2021SurfEmb: Dense and Continuous Correspondence Distributions for Object Pose Estimation with Learnt Surface Embeddingscited by 95
  16. 2021BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Modelscited by 87
  17. 2023RelPose++: Recovering 6D Poses from Sparse-view Observationscited by 87
  18. 2022FS6D: Few-Shot 6D Pose Estimation of Novel Objectscited by 81
  19. 2022OSOP: A Multi-Stage One Shot Object Pose Estimation Frameworkcited by 81
  20. 2021OMNet: Learning Overlapping Mask for Partial-to-Partial Point Cloud Registrationcited by 51
  21. 2022OVE6D: Object Viewpoint Encoding for Depth-based 6D Object Pose Estimationcited by 43
  22. 2021Learning general and distinctive 3D local deep descriptors for point cloud registrationcited by 42
  23. 2023NOPE: Novel Object Pose Estimation from a Single Imagecited by 38
  24. 2020Distinctive 3D local deep descriptorscited by 21
  25. 2022Learning 6-DoF Object Poses to Grasp Category-level Objects by Language Instructionscited by 19
  26. 2022PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Trackingcited by 15
  27. 2022ObjectMatch: Robust Registration using Canonical Object Correspondencescited by 15
  28. 2023Revisiting Fully Convolutional Geometric Features for Object 6D Pose Estimationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 13; 2026: 10.

29 papers.

Sort:

By area:

  1. 2025Any6D: Model-free 6D Pose Estimation of Novel Objectscited by 49
  2. 2025SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulationcited by 46
  3. 2024Deep Learning-Based Object Pose Estimation: A Comprehensive Surveycited by 40
  4. 2024ImageNet3D: Towards General-Purpose Object-Level 3D Understandingcited by 25
  5. 2024F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactionscited by 24
  6. 2025GCE-Pose: Global Context Enhancement for Category-level Object Pose Estimationcited by 18
  7. 2025One2Any: One-Reference 6D Pose Estimation for Any Objectcited by 18
  8. 2024CLIPose: Category-Level Object Pose Estimation with Pre-trained Vision-Language Knowledgecited by 16
  9. 2025Accurate and efficient zero-shot 6D pose estimation with frozen foundation modelscited by 13
  10. 2024UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Imagecited by 12
  11. 2025Scalable Unseen Objects 6-DoF Absolute Pose Estimation with Robotic Integrationcited by 11
  12. 2024High-resolution open-vocabulary object 6D pose estimationcited by 10
17 more
  1. 2025Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perceptioncited by 10
  2. 2025ConceptPose: Training-Free Zero-Shot Object Pose Estimation using Concept Vectorscited by 5
  3. 2026ProxyPose: 6-DoF Pose Tracking via Video-to-Video Translationcited by 3
  4. 2025UnPose: Uncertainty-Guided Diffusion Priors for Zero-Shot Pose Estimationcited by 2
  5. 2025Zero-shot Inexact CAD Model Alignment from a Single Imagecited by 1
  6. 2025SingRef6D: Monocular Novel Object Pose Estimation with a Single RGB Referencecited by 1
  7. 2025OPFormer: Object Pose Estimation leveraging foundation model with geometric encodingcited by 1
  8. 2026GenHOI: Generalized Hand-Object Pose Estimation with Occlusion Awarenesscited by 1
  9. 2025Color-Pair Guided Robust Zero-Shot 6D Pose Estimation and Tracking of Cluttered Objects on Edge Devicescited by 0
  10. 2026Learning Fine-Grained Correspondence with Cross-Perspective Perception for Open-Vocabulary 6D Object Pose Estimationcited by 0
  11. 2026Object Pose Transformer: Unifying Unseen Object Pose Estimationcited by 0
  12. 2026MAPRPose: Mask-Aware Proposal and Amodal Refinement for Multi-Object 6D Pose Estimationcited by 0
  13. 2026Semantic Prior Guided One-View 6D Pose Estimation for Novel Objectscited by 0
  14. 2026Learning Cross-View Semantic Priors for Single-Reference Unseen Object Pose Estimationcited by 0
  15. 2026Pose Anything Anywhere:Model-free Object Poses from Arbitrary Referencescited by 0
  16. 2026UniPose9D: Universal Category-Agnostic Object Pose Estimationcited by 0
  17. 2026Prior-free relative 6D pose estimation of multiple object instancescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024From Words to Poses: Enhancing Novel Object Pose Estimation with Vision Language Models
  2. 2024OV9D: Open-Vocabulary Category-Level 9D Object Pose and Size Estimation
  3. 2023Challenges for Monocular 6D Object Pose Estimation in Robotics
  4. 2024CapeLLM: Support-Free Category-Agnostic Pose Estimation with Multimodal Large Language Models
  5. 2025Beyond 'Templates': Category-Agnostic Object Pose, Size, and Shape Estimation from a Single View
  6. 2024Category Level 6D Object Pose Estimation from a Single RGB Image using Diffusion
  7. 2026OMNI-PoseX: A Fast Vision Model for 6D Object Pose Estimation in Embodied Tasks ↗
  8. 2022Knowledge Distillation for 6D Pose Estimation by Aligning Distributions of Local Predictions