mlsanityData to 7 Oct 2026

Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning

arXiv 1807.03146 ↗First posted 5 Jul 2018cs.CV cs.LG stat.MLcited by 79

Not on the map. Why?

Abstract (arXiv)

This paper presents KeypointNet, an end-to-end geometric reasoning framework to learn an optimal set of category-specific 3D keypoints, along with their detectors. Given a single image, KeypointNet extracts 3D keypoints that are optimized for a downstream task. We demonstrate this framework on 3D pose estimation by proposing a differentiable objective that seeks the optimal set of keypoints for recovering the relative pose between two views of an object. Our model discovers geometrically and semantically consistent keypoints across viewing angles and instances of an object category. Importantly, we find that our end-to-end framework using no ground-truth keypoint annotations outperforms a fully supervised baseline using the same neural network architecture on the task of pose estimation. The discovered 3D keypoints on the car, chair, and plane categories of ShapeNet are visualized at http://keypointnet.github.io/.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 33; 2022: 23; 2023: 13; 2024: 6; 2025: 2; 2026: 2.

79 papers.

Sort:

By area:

  1. 2021PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clusteringcited by 163
  2. 2022Map-free Visual Relocalization: Metric Pose Relative to a Single Imagecited by 112
  3. 2021Repurposing GANs for One-shot Semantic Part Segmentationcited by 88
  4. 2021BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Modelscited by 87
  5. 2021LOLNeRF: Learn from One Lookcited by 84
  6. 2021Implicit-PDF: Non-Parametric Representation of Probability Distributions on the Rotation Manifoldcited by 65
  7. 2021KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Controlcited by 50
  8. 2021Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objectscited by 50
  9. 2021Wide-Baseline Relative Camera Pose Estimation with Directional Learningcited by 49
  10. 2021Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learningcited by 49
  11. 2021StereOBJ-1M: Large-scale Stereo Image Dataset for 6D Object Pose Estimationcited by 40
  12. 2023Improving Semantic Correspondence with Viewpoint-Guided Spherical Mapscited by 33
38 more
  1. 2021Supervision by Registration and Triangulation for Landmark Detectioncited by 30
  2. 2021Unsupervised Learning of Visual 3D Keypoints for Controlcited by 30
  3. 2022USEEK: Unsupervised SE(3)-Equivariant 3D Keypoints for Generalizable Manipulationcited by 27
  4. 2021Two-Stage Mesh Deep Learning for Automated Tooth Segmentation and Landmark Localization on 3D Intraoral Scanscited by 23
  5. 2022Vision+X: A Survey on Multimodal Learning in the Light of Datacited by 22
  6. 2021Rethinking Keypoint Representations: Modeling Keypoints and Poses as Objects for Multi-Person Human Pose Estimationcited by 20
  7. 20233D Implicit Transporter for Temporally Consistent Keypoint Discoverycited by 20
  8. 2021GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generationcited by 19
  9. 2021StablePose: Learning 6D Object Poses from Geometrically Stable Patchescited by 17
  10. 2021Orienting Novel 3D Objects Using Self-Supervised Learning of Rotation Transformscited by 17
  11. 2022AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypointscited by 17
  12. 2021Optimal Pose and Shape Estimation for Category-level 3D Object Perceptioncited by 15
  13. 2021MonoCInIS: Camera Independent Monocular 3D Object Detection using Instance Segmentationcited by 15
  14. 2021Pushing it out of the Way: Interactive Visual Navigationcited by 14
  15. 2022Filtered-CoPhy: Unsupervised Learning of Counterfactual Physics in Pixel Spacecited by 14
  16. 2023Point2SSM: Learning Morphological Variations of Anatomies from Point Cloudcited by 14
  17. 2024Learning to Make Keypoints Sub-Pixel Accuratecited by 13
  18. 2021ASK: Adaptively Selecting Key Local Features for RGB-D Scene Recognitioncited by 12
  19. 2022Certifiable 3D Object Pose Estimation: Foundations, Learning Models, and Self-Trainingcited by 12
  20. 2022Structural Causal 3D Reconstructioncited by 12
  21. 2022HoW-3D: Holistic 3D Wireframe Perception from a Single Imagecited by 12
  22. 2022SNAKE: Shape-aware Neural 3D Keypoint Fieldcited by 10
  23. 2021KDFNet: Learning Keypoint Distance Field for 6D Object Pose Estimationcited by 9
  24. 20233D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labelscited by 9
  25. 2023SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Datacited by 9
  26. 2022Optimal and Robust Category-level Perception: Object Pose and Shape Estimation from 2D and 3D Semantic Keypointscited by 6
  27. 2023Few-shot Geometry-Aware Keypoint Localizationcited by 6
  28. 2021LatentKeypointGAN: Controlling Images via Latent Keypointscited by 5
  29. 2021Semi-supervised Viewpoint Estimation with Geometry-aware Conditional Generationcited by 5
  30. 2021Vision-based system identification and 3D keypoint discovery using dynamics constraintscited by 5
  31. 2024Point2SSM++: Self-Supervised Learning of Anatomical Shape Models from Point Cloudscited by 5
  32. 2021Tight Integration of Feature-based Relocalization in Monocular Direct Visual Odometrycited by 4
  33. 2021Kit-Net: Self-Supervised Learning to Kit Novel 3D Objects into Novel 3D Cavitiescited by 4
  34. 2024Key-Grid: Unsupervised 3D Keypoints Detection using Grid Heatmap Featurescited by 4
  35. 2021Learning ABCs: Approximate Bijective Correspondence for isolating factors of variation with weak supervisioncited by 3
  36. 2021JOKR: Joint Keypoint Representation for Unsupervised Cross-Domain Motion Retargetingcited by 3
  37. 2021Single Image 3D Object Estimation with Primitive Graph Networkscited by 3
  38. 2021Few-Shot Keypoint Detection as Task Adaptation via Latent Embeddingscited by 3

50 most cited of 79 All 79 (JSON)