# Unsupervised Learning of Category-Specific Symmetric 3D Keypoints from Point Sets

arXiv: https://arxiv.org/abs/2003.07619 · HTML: https://mlsanity.com/paper/2003.07619/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2020.

## Built on by (most cited first, 23 of 23)

- [DOVE: Learning Deformable 3D Objects by Watching Videos](https://mlsanity.com/paper/2107.10844/) (2021)
- [KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control](https://mlsanity.com/paper/2104.11224/) (2021)
- [Skeleton-free Pose Transfer for Stylized 3D Characters](https://mlsanity.com/paper/2208.00790/) (2022)
- [Skeleton Merger: an Unsupervised Aligned Keypoint Detector](https://mlsanity.com/paper/2103.10814/) (2021)
- [LAKe-Net: Topology-Aware Point Cloud Completion by Localizing Aligned Keypoints](https://mlsanity.com/paper/2203.16771/) (2022)
- [Two-Stage Mesh Deep Learning for Automated Tooth Segmentation and Landmark Localization on 3D Intraoral Scans](https://mlsanity.com/paper/2109.11941/) (2021)
- [Point2SSM: Learning Morphological Variations of Anatomies from Point Cloud](https://mlsanity.com/paper/2305.14486/) (2023)
- [OMAD: Object Model with Articulated Deformations for Pose Estimation and Retrieval](https://mlsanity.com/paper/2112.07334/) (2021)
- [SNAKE: Shape-aware Neural 3D Keypoint Field](https://mlsanity.com/paper/2206.01724/) (2022)
- [SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data](https://mlsanity.com/paper/2308.05410/) (2023)
- [Representing Shape Collections with Alignment-Aware Linear Models](https://mlsanity.com/paper/2109.01605/) (2021)
- [LatentKeypointGAN: Controlling Images via Latent Keypoints](https://mlsanity.com/paper/2103.15812/) (2021)
- [Unsupervised Learning of 3D Semantic Keypoints with Mutual Reconstruction](https://mlsanity.com/paper/2203.10212/) (2022)
- [Point2SSM++: Self-Supervised Learning of Anatomical Shape Models from Point Clouds](https://mlsanity.com/paper/2405.09707/) (2024)
- [Unsupervised 3D Keypoint Discovery with Multi-View Geometry](https://mlsanity.com/paper/2211.12829/) (2022)
- [Learning Canonical Embedding for Non-rigid Shape Matching](https://mlsanity.com/paper/2110.02994/) (2021)
- [Semantic keypoint extraction for scanned animals using multi-depth-camera systems](https://mlsanity.com/paper/2211.08634/) (2022)
- [3D Keypoint Estimation Using Implicit Representation Learning](https://mlsanity.com/paper/2306.11529/) (2023)
- [SelfGeo: Self-supervised and Geodesic-consistent Estimation of Keypoints on Deformable Shapes](https://mlsanity.com/paper/2408.02291/) (2024)
- [Joint Symmetry Detection and Shape Matching for Non-Rigid Point Cloud](https://mlsanity.com/paper/2112.02713/) (2021)
- [LatentKeypointGAN: Controlling Images via Latent Keypoints -- Extended Abstract](https://mlsanity.com/paper/2205.03448/) (2022)
- [Exploring Category-level Articulated Object Pose Tracking on SE(3) Manifolds](https://mlsanity.com/paper/2511.05996/) (2025)
- [KeyPointDiffuser: Unsupervised 3D Keypoint Learning via Latent Diffusion Models](https://mlsanity.com/paper/2512.03450/) (2025)

## Abstract (arXiv)

Automatic discovery of category-specific 3D keypoints from a collection of objects of some category is a challenging problem. One reason is that not all objects in a category necessarily have the same semantic parts. The level of difficulty adds up further when objects are represented by 3D point clouds, with variations in shape and unknown coordinate frames. We define keypoints to be category-specific, if they meaningfully represent objects' shape and their correspondences can be simply established order-wise across all objects. This paper aims at learning category-specific 3D keypoints, in an unsupervised manner, using a collection of misaligned 3D point clouds of objects from an unknown category. In order to do so, we model shapes defined by the keypoints, within a category, using the symmetric linear basis shapes without assuming the plane of symmetry to be known. The usage of symmetry prior leads us to learn stable keypoints suitable for higher misalignments. To the best of our knowledge, this is the first work on learning such keypoints directly from 3D point clouds. Using categories from four benchmark datasets, we demonstrate the quality of our learned keypoints by quantitative and qualitative evaluations. Our experiments also show that the keypoints discovered by our method are geometrically and semantically consistent.