# SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data

arXiv: https://arxiv.org/abs/2308.05410 · HTML: https://mlsanity.com/paper/2308.05410/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 47 references that are in the map.
- First posted on arXiv 10 Aug 2023.

## Builds on (most cited first, 30 of 31)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018)
- [PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1911.04231/) (2019)
- [PRNet: Self-Supervised Learning for Partial-to-Partial Registration](https://mlsanity.com/paper/1910.12240/) (2019)
- [SCOPS: Self-Supervised Co-Part Segmentation](https://mlsanity.com/paper/1905.01298/) (2019)
- [Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning](https://mlsanity.com/paper/1807.03146/) (2018)
- [KeyPose: Multi-View 3D Labeling and Keypoint Estimation for Transparent Objects](https://mlsanity.com/paper/1912.02805/) (2019)
- [Self-supervised Learning of Point Clouds via Orientation Estimation](https://mlsanity.com/paper/2008.00305/) (2020)
- [USIP: Unsupervised Stable Interest Point Detection from 3D Point Clouds](https://mlsanity.com/paper/1904.00229/) (2019)
- [Unsupervised Part-Based Disentangling of Object Shape and Appearance](https://mlsanity.com/paper/1903.06946/) (2019)
- [KeypointNet: A Large-scale 3D Keypoint Dataset Aggregated from Numerous Human Annotations](https://mlsanity.com/paper/2002.12687/) (2020)
- [KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control](https://mlsanity.com/paper/2104.11224/) (2021)
- [Unsupervised Learning of Intrinsic Structural Representation Points](https://mlsanity.com/paper/2003.01661/) (2020)
- [ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes](https://mlsanity.com/paper/2201.07788/) (2022)
- [Skeleton Merger: an Unsupervised Aligned Keypoint Detector](https://mlsanity.com/paper/2103.10814/) (2021)
- [KAMA: 3D Keypoint Aware Body Mesh Articulation](https://mlsanity.com/paper/2104.13502/) (2021)
- [LAKe-Net: Topology-Aware Point Cloud Completion by Localizing Aligned Keypoints](https://mlsanity.com/paper/2203.16771/) (2022)
- [USEEK: Unsupervised SE(3)-Equivariant 3D Keypoints for Generalizable Manipulation](https://mlsanity.com/paper/2209.13864/) (2022)
- [Unsupervised Learning of Category-Specific Symmetric 3D Keypoints from Point Sets](https://mlsanity.com/paper/2003.07619/) (2020)
- [Learning 3D Dense Correspondence via Canonical Point Autoencoder](https://mlsanity.com/paper/2107.04867/) (2021)
- [UKPGAN: A General Self-Supervised Keypoint Detector](https://mlsanity.com/paper/2011.11974/) (2020)
- [OMAD: Object Model with Articulated Deformations for Pose Estimation and Retrieval](https://mlsanity.com/paper/2112.07334/) (2021)
- [SNAKE: Shape-aware Neural 3D Keypoint Field](https://mlsanity.com/paper/2206.01724/) (2022)
- [Unsupervised Point Cloud Pre-Training via Contrasting and Clustering](https://mlsanity.com/paper/2202.02543/) (2022)
- [Self-supervised Point Set Local Descriptors for Point Cloud Registration](https://mlsanity.com/paper/2003.05199/) (2020)
- [Task-Aware Sampling Layer for Point-Wise Analysis](https://mlsanity.com/paper/2107.04291/) (2021)
- [Unsupervised Learning of 3D Semantic Keypoints with Mutual Reconstruction](https://mlsanity.com/paper/2203.10212/) (2022)

## Built on by (most cited first, 9 of 9)

- [Point2SSM++: Self-Supervised Learning of Anatomical Shape Models from Point Clouds](https://mlsanity.com/paper/2405.09707/) (2024)
- [Key-Grid: Unsupervised 3D Keypoints Detection using Grid Heatmap Features](https://mlsanity.com/paper/2410.02237/) (2024)
- [Learning SO(3)-Invariant Semantic Correspondence via Local Shape Transform](https://mlsanity.com/paper/2404.11156/) (2024)
- [SelfGeo: Self-supervised and Geodesic-consistent Estimation of Keypoints on Deformable Shapes](https://mlsanity.com/paper/2408.02291/) (2024)
- [PAOLI: Pose-free Articulated Object Learning from Sparse-view Images](https://mlsanity.com/paper/2509.04276/) (2025)
- [KeyPointDiffuser: Unsupervised 3D Keypoint Learning via Latent Diffusion Models](https://mlsanity.com/paper/2512.03450/) (2025)
- [SCALP: Semi-Supervised Statistical Shape Modeling from Imperfect 3D Photogrammetry via Landmark-Anchored Spectral Warp](https://mlsanity.com/paper/2608.00187/) (2026)
- [TRACE: Artifact-Robust Statistical Shape Modeling from Imperfect Surface Scans - A Case Study in Craniosynostosis 3D Photography](https://mlsanity.com/paper/2608.22131/) (2026)
- [KeyGen: Unsupervised Keypoint based Object-Centric Representations for Category-Level Policy Generalization](https://mlsanity.com/paper/2609.28818/) (2026)

## Abstract (arXiv)

This paper proposes a new method to infer keypoints from arbitrary object categories in practical scenarios where point cloud data (PCD) are noisy, down-sampled and arbitrarily rotated. Our proposed model adheres to the following principles: i) keypoints inference is fully unsupervised (no annotation given), ii) keypoints position error should be low and resilient to PCD perturbations (robustness), iii) keypoints should not change their indexes for the intra-class objects (semantic coherence), iv) keypoints should be close to or proximal to PCD surface (compactness). We achieve these desiderata by proposing a new self-supervised training strategy for keypoints estimation that does not assume any a priori knowledge of the object class, and a model architecture with coupled auxiliary losses that promotes the desired keypoints properties. We compare the keypoints estimated by the proposed approach with those of the state-of-the-art unsupervised approaches. The experiments show that our approach outperforms by estimating keypoints with improved coverage (+9.41%) while being semantically consistent (+4.66%) that best characterizes the object's 3D shape for downstream tasks. Code and data are available at: https://github.com/IITPAVIS/SC3K