# KeyMatchNet: Zero-Shot Pose Estimation in 3D Point Clouds by Generalized Keypoint Matching

arXiv: https://arxiv.org/abs/2303.16102 · HTML: https://mlsanity.com/paper/2303.16102/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 42 references that are in the map.
- First posted on arXiv 28 Mar 2023.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1812.11788/) (2018)
- [DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion](https://mlsanity.com/paper/1901.04780/) (2019)
- [BOP: Benchmark for 6D Object Pose Estimation](https://mlsanity.com/paper/1808.08319/) (2018)
- [SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again](https://mlsanity.com/paper/1711.10006/) (2017)
- [BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth](https://mlsanity.com/paper/1703.10896/) (2017)
- [Real-Time Seamless Single Shot 6D Object Pose Prediction](https://mlsanity.com/paper/1711.08848/) (2017)
- [PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1911.04231/) (2019)
- [MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare](https://mlsanity.com/paper/2212.06870/) (2022)
- [BOP Challenge 2020 on 6D Object Localization](https://mlsanity.com/paper/2009.07378/) (2020)
- [DPOD: 6D Pose Object Detector and Refiner](https://mlsanity.com/paper/1902.11020/) (2019)
- [EPOS: Estimating 6D Pose of Objects with Symmetries](https://mlsanity.com/paper/2004.00605/) (2020)
- [FS6D: Few-Shot 6D Pose Estimation of Novel Objects](https://mlsanity.com/paper/2203.14628/) (2022)
- [OSOP: A Multi-Stage One Shot Object Pose Estimation Framework](https://mlsanity.com/paper/2203.15533/) (2022)
- [BlenderProc](https://mlsanity.com/paper/1911.01911/) (2019)
- [BOP Challenge 2022 on Detection, Segmentation and Pose Estimation of Specific Rigid Objects](https://mlsanity.com/paper/2302.13075/) (2023)
- [Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions](https://mlsanity.com/paper/2203.17234/) (2022)
- [PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking](https://mlsanity.com/paper/2209.07589/) (2022)
- [Unseen Object 6D Pose Estimation: A Benchmark and Baselines](https://mlsanity.com/paper/2206.11808/) (2022)
- [Rotational Subgroup Voting and Pose Clustering for Robust 3D Object Recognition](https://mlsanity.com/paper/1709.02142/) (2017)
- [Bridging the Reality Gap for Pose Estimation Networks using Sensor-Based Domain Randomization](https://mlsanity.com/paper/2011.08517/) (2020)
- [ParaPose: Parameter and Domain Randomization Optimization for Pose Estimation using Synthetic Data](https://mlsanity.com/paper/2203.00945/) (2022)
- [In-Hand Pose Estimation and Pin Inspection for Insertion of Through-Hole Components](https://mlsanity.com/paper/2208.01284/) (2022)

## Built on by (most cited first, 7 of 7)

- [Deep Learning-Based Object Pose Estimation: A Comprehensive Survey](https://mlsanity.com/paper/2405.07801/) (2024)
- [Scalable Unseen Objects 6-DoF Absolute Pose Estimation with Robotic Integration](https://mlsanity.com/paper/2503.05578/) (2025)
- [Good Grasps Only: A data engine for self-supervised fine-tuning of pose estimation using grasp poses for verification](https://mlsanity.com/paper/2409.11512/) (2024)
- [Foundation Feature-Driven Online End-Effector Pose Estimation: A Marker-Free and Learning-Free Approach](https://mlsanity.com/paper/2503.14051/) (2025)
- [SingRef6D: Monocular Novel Object Pose Estimation with a Single RGB Reference](https://mlsanity.com/paper/2509.21927/) (2025)
- [Object Pose Distribution Estimation for Determining Revolution and Reflection Uncertainty in Point Clouds](https://mlsanity.com/paper/2512.07211/) (2025)
- [Towards a Modular Bin-picking Framework for Handling Object Pose Uncertainties](https://mlsanity.com/paper/2607.13698/) (2026)

## Abstract (arXiv)

In this paper, we present KeyMatchNet, a novel network for zero-shot pose estimation in 3D point clouds. Our method uses only depth information, making it more applicable for many industrial use cases, as color information is seldom available. The network is composed of two parallel components for computing object and scene features. The features are then combined to create matches used for pose estimation. The parallel structure allows for pre-processing of the individual parts, which decreases the run-time. Using a zero-shot network allows for a very short set-up time, as it is not necessary to train models for new objects. However, as the network is not trained for the specific object, zero-shot pose estimation methods generally have lower accuracy compared with conventional methods. To address this, we reduce the complexity of the task by including the scenario information during training. This is typically not feasible as collecting real data for new tasks drastically increases the cost. However, for zero-shot pose estimation, training for new objects is not necessary and the expensive data collection can thus be performed only once. Our method is trained on 1,500 objects and is only tested on unseen objects. We demonstrate that the trained network can not only accurately estimate poses for novel objects, but also demonstrate the ability of the network on objects outside of the trained class. Test results are also shown on real data. We believe that the presented method is valuable for many real-world scenarios. Project page available at keymatchnet.github.io