# PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation

arXiv: https://arxiv.org/abs/1911.04231 · HTML: https://mlsanity.com/paper/1911.04231/

## Key facts

- Cited by 204 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 30 of 204)

- [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023)
- [DexMV: Imitation Learning for Dexterous Manipulation from Human Videos](https://mlsanity.com/paper/2108.05877/) (2021)
- [GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimator](https://mlsanity.com/paper/2102.12145/) (2021)
- [FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation](https://mlsanity.com/paper/2103.02242/) (2021)
- [PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency](https://mlsanity.com/paper/2103.05465/) (2021)
- [Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation](https://mlsanity.com/paper/2405.01527/) (2024)
- [FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanism](https://mlsanity.com/paper/2103.07054/) (2021)
- [SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation](https://mlsanity.com/paper/2311.15707/) (2023)
- [GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting](https://mlsanity.com/paper/2203.07918/) (2022)
- [BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Models](https://mlsanity.com/paper/2108.00516/) (2021)
- [FS6D: Few-Shot 6D Pose Estimation of Novel Objects](https://mlsanity.com/paper/2203.14628/) (2022)
- [Full-Body Articulated Human-Object Interaction](https://mlsanity.com/paper/2212.10621/) (2022)
- [Category-Level 6D Object Pose and Size Estimation using Self-Supervised Deep Prior Deformation Networks](https://mlsanity.com/paper/2207.05444/) (2022)
- [Category-Level 6D Object Pose Estimation via Cascaded Relation and Recurrent Reconstruction Networks](https://mlsanity.com/paper/2108.08755/) (2021)
- [Deep Regression on Manifolds: A 3D Rotation Case Study](https://mlsanity.com/paper/2103.16317/) (2021)
- [Category-Level 6D Object Pose Estimation in the Wild: A Semi-Supervised Learning Approach and A New Dataset](https://mlsanity.com/paper/2206.15436/) (2022)
- [Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation](https://mlsanity.com/paper/2210.10108/) (2022)
- [SAR-Net: Shape Alignment and Recovery Network for Category-level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/2106.14193/) (2021)
- [SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimation](https://mlsanity.com/paper/2311.11125/) (2023)
- [Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Plans](https://mlsanity.com/paper/2312.00775/) (2023)
- [Zero-Shot Robot Manipulation from Passive Human Videos](https://mlsanity.com/paper/2302.02011/) (2023)
- [Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects](https://mlsanity.com/paper/2208.03792/) (2022)
- [HS-Pose: Hybrid Scope Feature Extraction for Category-level Object Pose Estimation](https://mlsanity.com/paper/2303.15743/) (2023)
- [Any6D: Model-free 6D Pose Estimation of Novel Objects](https://mlsanity.com/paper/2503.18673/) (2025)
- [Coupled Iterative Refinement for 6D Multi-Object Pose Estimation](https://mlsanity.com/paper/2204.12516/) (2022)
- [TAX-Pose: Task-Specific Cross-Pose Estimation for Robot Manipulation](https://mlsanity.com/paper/2211.09325/) (2022)
- [OVE6D: Object Viewpoint Encoding for Depth-based 6D Object Pose Estimation](https://mlsanity.com/paper/2203.01072/) (2022)
- [RBP-Pose: Residual Bounding Box Projection for Category-Level Pose Estimation](https://mlsanity.com/paper/2208.00237/) (2022)
- [Instance-Adaptive and Geometric-Aware Keypoint Learning for Category-Level 6D Object Pose Estimation](https://mlsanity.com/paper/2403.19527/) (2024)
- [SSP-Pose: Symmetry-Aware Shape Prior Deformation for Direct Category-Level Object Pose Estimation](https://mlsanity.com/paper/2208.06661/) (2022)

## Abstract (arXiv)

In this work, we present a novel data-driven method for robust 6DoF object pose estimation from a single RGBD image. Unlike previous methods that directly regressing pose parameters, we tackle this challenging task with a keypoint-based approach. Specifically, we propose a deep Hough voting network to detect 3D keypoints of objects and then estimate the 6D pose parameters within a least-squares fitting manner. Our method is a natural extension of 2D-keypoint approaches that successfully work on RGB based 6DoF estimation. It allows us to fully utilize the geometric constraint of rigid objects with the extra depth information and is easy for a network to learn and optimize. Extensive experiments were conducted to demonstrate the effectiveness of 3D-keypoint detection in the 6D pose estimation task. Experimental results also show our method outperforms the state-of-the-art methods by large margins on several benchmarks. Code and video are available at https://github.com/ethnhe/PVN3D.git.