# ZeroPose: CAD-Prompted Zero-shot Object 6D Pose Estimation in Cluttered Scenes

arXiv: https://arxiv.org/abs/2305.17934 · HTML: https://mlsanity.com/paper/2305.17934/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 68 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 30 of 38)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion](https://mlsanity.com/paper/1901.04780/) (2019)
- [CosyPose: Consistent multi-view multi-object 6D pose estimation](https://mlsanity.com/paper/2008.08465/) (2020)
- [T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objects](https://mlsanity.com/paper/1701.05498/) (2017)
- [MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare](https://mlsanity.com/paper/2212.06870/) (2022)
- [BOP Challenge 2020 on 6D Object Localization](https://mlsanity.com/paper/2009.07378/) (2020)
- [OnePose: One-Shot Object Pose Estimation without CAD Models](https://mlsanity.com/paper/2205.12257/) (2022)
- [ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimation](https://mlsanity.com/paper/2203.09418/) (2022)
- [OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Models](https://mlsanity.com/paper/2301.07673/) (2023)
- [SurfEmb: Dense and Continuous Correspondence Distributions for Object Pose Estimation with Learnt Surface Embeddings](https://mlsanity.com/paper/2111.13489/) (2021)
- [LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation](https://mlsanity.com/paper/1912.00416/) (2019)
- [RelPose++: Recovering 6D Poses from Sparse-view Observations](https://mlsanity.com/paper/2305.04926/) (2023)
- [Unseen Object Instance Segmentation for Robotic Environments](https://mlsanity.com/paper/2007.08073/) (2020)
- [FS6D: Few-Shot 6D Pose Estimation of Novel Objects](https://mlsanity.com/paper/2203.14628/) (2022)
- [OSOP: A Multi-Stage One Shot Object Pose Estimation Framework](https://mlsanity.com/paper/2203.15533/) (2022)
- [BlenderProc](https://mlsanity.com/paper/1911.01911/) (2019)
- [Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation](https://mlsanity.com/paper/2007.15157/) (2020)
- [Understanding Humans in Crowded Scenes: Deep Nested Adversarial Learning and A New Benchmark for Multi-Human Parsing](https://mlsanity.com/paper/1804.03287/) (2018)
- [Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions](https://mlsanity.com/paper/2203.17234/) (2022)
- [Coupled Iterative Refinement for 6D Multi-Object Pose Estimation](https://mlsanity.com/paper/2204.12516/) (2022)
- [OVE6D: Object Viewpoint Encoding for Depth-based 6D Object Pose Estimation](https://mlsanity.com/paper/2203.01072/) (2022)

## Built on by (most cited first, 20 of 20)

- [FoundPose: Unseen Object Pose Estimation with Foundation Features](https://mlsanity.com/paper/2311.18809/) (2023)
- [Any6D: Model-free 6D Pose Estimation of Novel Objects](https://mlsanity.com/paper/2503.18673/) (2025)
- [Accurate and efficient zero-shot 6D pose estimation with frozen foundation models](https://mlsanity.com/paper/2506.09784/) (2025)
- [UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Image](https://mlsanity.com/paper/2411.16106/) (2024)
- [6DOPE-GS: Online 6D Object Pose Estimation using Gaussian Splatting](https://mlsanity.com/paper/2412.01543/) (2024)
- [UA-Pose: Uncertainty-Aware 6D Object Pose Estimation and Online Object Completion with Partial References](https://mlsanity.com/paper/2506.07996/) (2025)
- [BOP-Distrib: Revisiting 6D Pose Estimation Benchmarks for Better Evaluation under Visual Ambiguities](https://mlsanity.com/paper/2408.17297/) (2024)
- [NOCTIS: Novel Object Cyclic Threshold based Instance Segmentation](https://mlsanity.com/paper/2507.01463/) (2025)
- [UnPose: Uncertainty-Guided Diffusion Priors for Zero-Shot Pose Estimation](https://mlsanity.com/paper/2508.15972/) (2025)
- [Finding NeMO: A Geometry-Aware Representation of Template Views for Few-Shot Perception](https://mlsanity.com/paper/2602.04343/) (2026)
- [Towards Human-Level 3D Relative Pose Estimation: Generalizable, Training-Free, with Single Reference](https://mlsanity.com/paper/2406.18453/) (2024)
- [DynOPETs: A Versatile Benchmark for Dynamic Object Pose Estimation and Tracking in Moving Camera Scenarios](https://mlsanity.com/paper/2503.19625/) (2025)
- [Color-Pair Guided Robust Zero-Shot 6D Pose Estimation and Tracking of Cluttered Objects on Edge Devices](https://mlsanity.com/paper/2509.23647/) (2025)
- [GS2POSE: Marry Gaussian Splatting to 6D Object Pose Estimation](https://mlsanity.com/paper/2510.16777/) (2025)
- [Geo6DPose: Fast Zero-Shot 6D Object Pose Estimation via Geometry-Filtered Feature Matching](https://mlsanity.com/paper/2512.10674/) (2025)
- [RFM-Pose:Reinforcement-Guided Flow Matching for Fast Category-Level 6D Pose Estimation](https://mlsanity.com/paper/2602.05257/) (2026)
- [MAPRPose: Mask-Aware Proposal and Amodal Refinement for Multi-Object 6D Pose Estimation](https://mlsanity.com/paper/2604.20650/) (2026)
- [Prior-free relative 6D pose estimation of multiple object instances](https://mlsanity.com/paper/2609.08949/) (2026)
- [G6D: Geometric Learning-Free RGB-D 6D Pose Solver for Robotic Manipulation](https://mlsanity.com/paper/2609.23566/) (2026)
- [Toward Comprehensive 3D Grounding: Orientation Grounding through Vision-Language Models](https://mlsanity.com/paper/2609.33109/) (2026)

## Abstract (arXiv)

Many robotics and industry applications have a high demand for the capability to estimate the 6D pose of novel objects from the cluttered scene. However, existing classic pose estimation methods are object-specific, which can only handle the specific objects seen during training. When applied to a novel object, these methods necessitate a cumbersome onboarding process, which involves extensive dataset preparation and model retraining. The extensive duration and resource consumption of onboarding limit their practicality in real-world applications. In this paper, we introduce ZeroPose, a novel zero-shot framework that performs pose estimation following a Discovery-Orientation-Registration (DOR) inference pipeline. This framework generalizes to novel objects without requiring model retraining. Given the CAD model of a novel object, ZeroPose enables in seconds onboarding time to extract visual and geometric embeddings from the CAD model as a prompt. With the prompting of the above embeddings, DOR can discover all related instances and estimate their 6D poses without additional human interaction or presupposing scene conditions. Compared with existing zero-shot methods solved by the render-and-compare paradigm, the DOR pipeline formulates the object pose estimation into a feature-matching problem, which avoids time-consuming online rendering and improves efficiency. Experimental results on the seven datasets show that ZeroPose as a zero-shot method achieves comparable performance with object-specific training methods and outperforms the state-of-the-art zero-shot method with 50x inference speed improvement.