# SelfGeo: Self-supervised and Geodesic-consistent Estimation of Keypoints on Deformable Shapes

arXiv: https://arxiv.org/abs/2408.02291 · HTML: https://mlsanity.com/paper/2408.02291/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 46 references that are in the map.
- First posted on arXiv 5 Aug 2024.

## Builds on (most cited first, 30 of 32)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose](https://mlsanity.com/paper/2102.06199/) (2021)
- [4DComplete: Non-Rigid Motion Estimation Beyond the Observable Surface](https://mlsanity.com/paper/2105.01905/) (2021)
- [Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning](https://mlsanity.com/paper/1807.03146/) (2018)
- [USIP: Unsupervised Stable Interest Point Detection from 3D Point Clouds](https://mlsanity.com/paper/1904.00229/) (2019)
- [LIMP: Learning Latent Shape Representations with Metric Preservation Priors](https://mlsanity.com/paper/2003.12283/) (2020)
- [KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control](https://mlsanity.com/paper/2104.11224/) (2021)
- [Unsupervised Learning of Visual 3D Keypoints for Control](https://mlsanity.com/paper/2106.07643/) (2021)
- [Skeleton Merger: an Unsupervised Aligned Keypoint Detector](https://mlsanity.com/paper/2103.10814/) (2021)
- [LAKe-Net: Topology-Aware Point Cloud Completion by Localizing Aligned Keypoints](https://mlsanity.com/paper/2203.16771/) (2022)
- [USEEK: Unsupervised SE(3)-Equivariant 3D Keypoints for Generalizable Manipulation](https://mlsanity.com/paper/2209.13864/) (2022)
- [Bending Graphs: Hierarchical Shape Matching using Gated Optimal Transport](https://mlsanity.com/paper/2202.01537/) (2022)
- [Unsupervised Learning of Category-Specific Symmetric 3D Keypoints from Point Sets](https://mlsanity.com/paper/2003.07619/) (2020)
- [ClothesNet: An Information-Rich 3D Garment Model Repository with Simulated Clothes Environment](https://mlsanity.com/paper/2308.09987/) (2023)
- [HUM3DIL: Semi-supervised Multi-modal 3D Human Pose Estimation for Autonomous Driving](https://mlsanity.com/paper/2212.07729/) (2022)
- [3D Implicit Transporter for Temporally Consistent Keypoint Discovery](https://mlsanity.com/paper/2309.05098/) (2023)
- [NCP: Neural Correspondence Prior for Effective Unsupervised Shape Matching](https://mlsanity.com/paper/2301.05839/) (2023)
- [UKPGAN: A General Self-Supervised Keypoint Detector](https://mlsanity.com/paper/2011.11974/) (2020)
- [PRIN/SPRIN: On Extracting Point-wise Rotation Invariant Features](https://mlsanity.com/paper/2102.12093/) (2021)
- [Continuous Geodesic Convolutions for Learning on 3D Shapes](https://mlsanity.com/paper/2002.02506/) (2020)
- [CoKe: Localized Contrastive Learning for Robust Keypoint Detection](https://mlsanity.com/paper/2009.14115/) (2020)
- [HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences](https://mlsanity.com/paper/2103.15573/) (2021)
- [SNAKE: Shape-aware Neural 3D Keypoint Field](https://mlsanity.com/paper/2206.01724/) (2022)
- [3D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labels](https://mlsanity.com/paper/2306.04745/) (2023)
- [SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data](https://mlsanity.com/paper/2308.05410/) (2023)
- [Object Wake-up: 3D Object Rigging from a Single Image](https://mlsanity.com/paper/2108.02708/) (2021)
- [Optimal and Robust Category-level Perception: Object Pose and Shape Estimation from 2D and 3D Semantic Keypoints](https://mlsanity.com/paper/2206.12498/) (2022)
- [Generalizable Local Feature Pre-training for Deformable Shape Analysis](https://mlsanity.com/paper/2303.15104/) (2023)

## Built on by (most cited first, 1 of 1)

- [KeyPointDiffuser: Unsupervised 3D Keypoint Learning via Latent Diffusion Models](https://mlsanity.com/paper/2512.03450/) (2025)

## Abstract (arXiv)

Unsupervised 3D keypoints estimation from Point Cloud Data (PCD) is a complex task, even more challenging when an object shape is deforming. As keypoints should be semantically and geometrically consistent across all the 3D frames - each keypoint should be anchored to a specific part of the deforming shape irrespective of intrinsic and extrinsic motion. This paper presents, "SelfGeo", a self-supervised method that computes persistent 3D keypoints of non-rigid objects from arbitrary PCDs without the need of human annotations. The gist of SelfGeo is to estimate keypoints between frames that respect invariant properties of deforming bodies. Our main contribution is to enforce that keypoints deform along with the shape while keeping constant geodesic distances among them. This principle is then propagated to the design of a set of losses which minimization let emerge repeatable keypoints in specific semantic locations of the non-rigid shape. We show experimentally that the use of geodesic has a clear advantage in challenging dynamic scenes and with different classes of deforming shapes (humans and animals). Code and data are available at: https://github.com/IIT-PAVIS/SelfGeo