# KeypointNet: A Large-scale 3D Keypoint Dataset Aggregated from Numerous Human Annotations

arXiv: https://arxiv.org/abs/2002.12687 · HTML: https://mlsanity.com/paper/2002.12687/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2020.

## Built on by (most cited first, 30 of 51)

- [Where2Act: From Pixels to Actions for Articulated 3D Objects](https://mlsanity.com/paper/2101.02692/) (2021)
- [PRA-Net: Point Relation-Aware Network for 3D Point Cloud Analysis](https://mlsanity.com/paper/2112.04903/) (2021)
- [KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control](https://mlsanity.com/paper/2104.11224/) (2021)
- [FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Models](https://mlsanity.com/paper/2303.12786/) (2023)
- [Neural Shape Mating: Self-Supervised Object Assembly with Adversarial Shape Priors](https://mlsanity.com/paper/2205.14886/) (2022)
- [Affordance Learning from Play for Sample-Efficient Policy Learning](https://mlsanity.com/paper/2203.00352/) (2022)
- [Skeleton Merger: an Unsupervised Aligned Keypoint Detector](https://mlsanity.com/paper/2103.10814/) (2021)
- [LAKe-Net: Topology-Aware Point Cloud Completion by Localizing Aligned Keypoints](https://mlsanity.com/paper/2203.16771/) (2022)
- [USEEK: Unsupervised SE(3)-Equivariant 3D Keypoints for Generalizable Manipulation](https://mlsanity.com/paper/2209.13864/) (2022)
- [Position: Measure Dataset Diversity, Don't Just Claim It](https://mlsanity.com/paper/2407.08188/) (2024)
- [Learning 3D Dense Correspondence via Canonical Point Autoencoder](https://mlsanity.com/paper/2107.04867/) (2021)
- [CoCoNets: Continuous Contrastive 3D Scene Representations](https://mlsanity.com/paper/2104.03851/) (2021)
- [NCP: Neural Correspondence Prior for Effective Unsupervised Shape Matching](https://mlsanity.com/paper/2301.05839/) (2023)
- [Back to 3D: Few-Shot 3D Keypoint Detection with Back-Projected 2D Features](https://mlsanity.com/paper/2311.18113/) (2023)
- [Certifiable 3D Object Pose Estimation: Foundations, Learning Models, and Self-Training](https://mlsanity.com/paper/2206.11215/) (2022)
- [SRIF: Semantic Shape Registration Empowered by Diffusion-based Image Morphing and Flow Estimation](https://mlsanity.com/paper/2409.11682/) (2024)
- [Localization with Sampling-Argmax](https://mlsanity.com/paper/2110.08825/) (2021)
- [Understanding Physical Dynamics with Counterfactual World Modeling](https://mlsanity.com/paper/2312.06721/) (2023)
- [SNAKE: Shape-aware Neural 3D Keypoint Field](https://mlsanity.com/paper/2206.01724/) (2022)
- [SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data](https://mlsanity.com/paper/2308.05410/) (2023)
- [Temporally-Coherent Surface Reconstruction via Metric-Consistent Atlases](https://mlsanity.com/paper/2104.06950/) (2021)
- [Optimal and Robust Category-level Perception: Object Pose and Shape Estimation from 2D and 3D Semantic Keypoints](https://mlsanity.com/paper/2206.12498/) (2022)
- [Task-Aware Sampling Layer for Point-Wise Analysis](https://mlsanity.com/paper/2107.04291/) (2021)
- [PatchAlign3D: Local Feature Alignment for Dense 3D Shape Understanding](https://mlsanity.com/paper/2601.02457/) (2026)
- [Unsupervised Learning of 3D Semantic Keypoints with Mutual Reconstruction](https://mlsanity.com/paper/2203.10212/) (2022)
- [Deep Deformable Models: Learning 3D Shape Abstractions with Part Consistency](https://mlsanity.com/paper/2309.01035/) (2023)
- [Key-Grid: Unsupervised 3D Keypoints Detection using Grid Heatmap Features](https://mlsanity.com/paper/2410.02237/) (2024)
- [CRIN: Rotation-Invariant Point Cloud Analysis and Rotation Estimation via Centrifugal Reference Frame](https://mlsanity.com/paper/2303.03101/) (2023)
- [Learning Markerless Robot-Depth Camera Calibration and End-Effector Pose Estimation](https://mlsanity.com/paper/2212.07567/) (2022)
- [Semantic-Aware Implicit Template Learning via Part Deformation Consistency](https://mlsanity.com/paper/2308.11916/) (2023)

## Abstract (arXiv)

Detecting 3D objects keypoints is of great interest to the areas of both graphics and computer vision. There have been several 2D and 3D keypoint datasets aiming to address this problem in a data-driven way. These datasets, however, either lack scalability or bring ambiguity to the definition of keypoints. Therefore, we present KeypointNet: the first large-scale and diverse 3D keypoint dataset that contains 103,450 keypoints and 8,234 3D models from 16 object categories, by leveraging numerous human annotations. To handle the inconsistency between annotations from different people, we propose a novel method to aggregate these keypoints automatically, through minimization of a fidelity loss. Finally, ten state-of-the-art methods are benchmarked on our proposed dataset. Our code and data are available on https://github.com/qq456cvb/KeypointNet.