mlsanityData to 7 Oct 2026

SelfGeo: Self-supervised and Geodesic-consistent Estimation of Keypoints on Deformable Shapes

arXiv 2408.02291 ↗First posted 5 Aug 2024cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Unsupervised 3D keypoints estimation from Point Cloud Data (PCD) is a complex task, even more challenging when an object shape is deforming. As keypoints should be semantically and geometrically consistent across all the 3D frames - each keypoint should be anchored to a specific part of the deforming shape irrespective of intrinsic and extrinsic motion. This paper presents, "SelfGeo", a self-supervised method that computes persistent 3D keypoints of non-rigid objects from arbitrary PCDs without the need of human annotations. The gist of SelfGeo is to estimate keypoints between frames that respect invariant properties of deforming bodies. Our main contribution is to enforce that keypoints deform along with the shape while keeping constant geodesic distances among them. This principle is then propagated to the design of a set of losses which minimization let emerge repeatable keypoints in specific semantic locations of the non-rigid shape. We show experimentally that the use of geodesic has a clear advantage in challenging dynamic scenes and with different classes of deforming shapes (humans and animals). Code and data are available at: https://github.com/IIT-PAVIS/SelfGeo

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 32 of its 46 references that are in the map. First posted on arXiv 5 Aug 2024.

Builds on

32 of 46 references.

References · 32

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  4. 2021A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Posecited by 233
  5. 20214DComplete: Non-Rigid Motion Estimation Beyond the Observable Surfacecited by 123
  6. 2018Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoningcited by 79
  7. 2019USIP: Unsupervised Stable Interest Point Detection from 3D Point Cloudscited by 70
  8. 2020LIMP: Learning Latent Shape Representations with Metric Preservation Priorscited by 54
  9. 2021KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Controlcited by 50
  10. 2021Unsupervised Learning of Visual 3D Keypoints for Controlcited by 30
  11. 2021Skeleton Merger: an Unsupervised Aligned Keypoint Detectorcited by 27
  12. 2022LAKe-Net: Topology-Aware Point Cloud Completion by Localizing Aligned Keypointscited by 27
20 more
  1. 2022USEEK: Unsupervised SE(3)-Equivariant 3D Keypoints for Generalizable Manipulationcited by 27
  2. 2022Bending Graphs: Hierarchical Shape Matching using Gated Optimal Transportcited by 24
  3. 2020Unsupervised Learning of Category-Specific Symmetric 3D Keypoints from Point Setscited by 23
  4. 2022HUM3DIL: Semi-supervised Multi-modal 3D Human Pose Estimation for Autonomous Drivingcited by 22
  5. 2023ClothesNet: An Information-Rich 3D Garment Model Repository with Simulated Clothes Environmentcited by 22
  6. 20233D Implicit Transporter for Temporally Consistent Keypoint Discoverycited by 20
  7. 2023NCP: Neural Correspondence Prior for Effective Unsupervised Shape Matchingcited by 18
  8. 2020UKPGAN: A General Self-Supervised Keypoint Detectorcited by 15
  9. 2021PRIN/SPRIN: On Extracting Point-wise Rotation Invariant Featurescited by 15
  10. 2020Continuous Geodesic Convolutions for Learning on 3D Shapescited by 14
  11. 2020CoKe: Localized Contrastive Learning for Robust Keypoint Detectioncited by 12
  12. 2021HumanGPS: Geodesic PreServing Feature for Dense Human Correspondencescited by 12
  13. 2022SNAKE: Shape-aware Neural 3D Keypoint Fieldcited by 10
  14. 20233D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labelscited by 9
  15. 2023SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Datacited by 9
  16. 2021Object Wake-up: 3D Object Rigging from a Single Imagecited by 8
  17. 2022Optimal and Robust Category-level Perception: Object Pose and Shape Estimation from 2D and 3D Semantic Keypointscited by 6
  18. 2023Generalizable Local Feature Pre-training for Deformable Shape Analysiscited by 6
  19. 2022Unsupervised Learning of 3D Semantic Keypoints with Mutual Reconstructioncited by 5
  20. 2023Semantic-Aware Implicit Template Learning via Part Deformation Consistencycited by 3

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025KeyPointDiffuser: Unsupervised 3D Keypoint Learning via Latent Diffusion Modelscited by 0