mlsanityData to 7 Oct 2026

KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control

arXiv 2104.11224 ↗First posted 22 Apr 2021cs.CV cs.GRcited by 50

Not on the map. Why?

Abstract (arXiv)

We introduce KeypointDeformer, a novel unsupervised method for shape control through automatically discovered 3D keypoints. We cast this as the problem of aligning a source 3D object to a target 3D object from the same object category. Our method analyzes the difference between the shapes of the two objects by comparing their latent representations. This latent representation is in the form of 3D keypoints that are learned in an unsupervised way. The difference between the 3D keypoints of the source and the target objects then informs the shape deformation algorithm that deforms the source object into the target object. The whole model is learned end-to-end and simultaneously discovers 3D keypoints while learning to use them for deforming object shapes. Our approach produces intuitive and semantically consistent control of shape deformations. Moreover, our discovered 3D keypoints are consistent across object category instances despite large shape variations. As our method is unsupervised, it can be readily deployed to new object categories without requiring annotations for 3D keypoints and deformations.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 19 of its 34 references that are in the map. First posted on arXiv 22 Apr 2021.

Builds on

19 of 34 references.

References · 19

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  3. 2016Learning Shape Abstractions by Assembling Volumetric Primitivescited by 181
  4. 2019kPAM: KeyPoint Affordances for Category-Level Robotic Manipulationcited by 152
  5. 2018Unsupervised Learning of Object Landmarks through Conditional Image Generationcited by 113
  6. 2019Neural Cages for Detail-Preserving 3D Deformationscited by 105
  7. 2020DualSDF: Semantic Shape Manipulation using a Two-Level Representationcited by 82
  8. 2018Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoningcited by 79
  9. 2018Unsupervised Discovery of Object Landmarks as Structural Representationscited by 73
  10. 2020Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learningcited by 69
  11. 2017Unsupervised learning of object landmarks by factorized spatial embeddingscited by 64
  12. 20193DN: 3D Deformation Networkcited by 60
7 more
  1. 2020KeypointNet: A Large-scale 3D Keypoint Dataset Aggregated from Numerous Human Annotationscited by 50
  2. 2018Self-supervised learning of a facial attribute embedding from videocited by 44
  3. 2019Unsupervised Learning of Landmarks by Descriptor Vector Exchangecited by 39
  4. 2020Unsupervised Learning of Intrinsic Structural Representation Pointscited by 39
  5. 2019kPAM-SC: Generalizable Manipulation Planning using KeyPoint Affordance and Shape Completioncited by 27
  6. 2020Learning Generative Models of Shape Handlescited by 25
  7. 2020Unsupervised Learning of Category-Specific Symmetric 3D Keypoints from Point Setscited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 13; 2023: 16; 2024: 8; 2025: 7; 2026: 6.

50 papers.

Sort:

By area:

  1. 2022Generative Neural Articulated Radiance Fieldscited by 101
  2. 2023TextDeformer: Geometry Manipulation using Text Guidancecited by 94
  3. 2022Deforming Radiance Fields with Cagescited by 78
  4. 2022Learning 3D Human Pose Estimation from Dozens of Datasets using a Geometry-Aware Autoencoder to Bridge Between Skeleton Formatscited by 50
  5. 2023NeuralEditor: Editing Neural Radiance Fields via Manipulating Point Cloudscited by 41
  6. 2023Controllable Mesh Generation Through Sparse Latent Point Diffusion Modelscited by 40
  7. 2022Skeleton-free Pose Transfer for Stylized 3D Characterscited by 29
  8. 2022Neural Jacobian Fields: Learning Intrinsic Mappings of Arbitrary Meshescited by 28
  9. 2022LAKe-Net: Topology-Aware Point Cloud Completion by Localizing Aligned Keypointscited by 27
  10. 2025Anymate: A Dataset and Baselines for Learning 3D Object Riggingcited by 27
  11. 2022Neural Shape Deformation Priorscited by 26
  12. 2023CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Sensor Simulationcited by 26
38 more
  1. 2023As-Plausible-As-Possible: Plausibility-Aware Mesh Deformation Using 2D Diffusion Priorscited by 24
  2. 2023NCP: Neural Correspondence Prior for Effective Unsupervised Shape Matchingcited by 18
  3. 2023Back to 3D: Few-Shot 3D Keypoint Detection with Back-Projected 2D Featurescited by 17
  4. 2023Point2SSM: Learning Morphological Variations of Anatomies from Point Cloudcited by 14
  5. 2023Weakly-supervised 3D Pose Transfer with Keypointscited by 12
  6. 2023Creative Birds: Self-Supervised Single-View 3D Style Transfercited by 9
  7. 2023SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Datacited by 9
  8. 2025Category-Agnostic Neural Object Riggingcited by 8
  9. 2022Structure-Aware 3D VR Sketch to 3D Shape Retrievalcited by 7
  10. 2023Point-Based Radiance Fields for Controllable Human Motion Synthesiscited by 7
  11. 2023ShapeMatcher: Self-Supervised Joint Shape Canonicalization, Segmentation, Retrieval and Deformationcited by 7
  12. 2023Spice-E : Structural Priors in 3D Diffusion using Cross-Entity Attentioncited by 7
  13. 2024Fancy123: One Image to High-Quality 3D Mesh Generation via Plug-and-Play Deformationcited by 7
  14. 2024KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformationcited by 6
  15. 2022Unsupervised Learning of 3D Semantic Keypoints with Mutual Reconstructioncited by 5
  16. 2024Point2SSM++: Self-Supervised Learning of Anatomical Shape Models from Point Cloudscited by 5
  17. 2022NeuralMLS: Geometry-Aware Control Point Deformationcited by 4
  18. 2024Key-Grid: Unsupervised 3D Keypoints Detection using Grid Heatmap Featurescited by 4
  19. 2022Learning Markerless Robot-Depth Camera Calibration and End-Effector Pose Estimationcited by 3
  20. 2023DragD3D: Realistic Mesh Editing with Rigidity Control Driven by 2D Diffusion Priorscited by 3
  21. 2023Towards Learning Geometric Eigen-Lengths Crucial for Fitting Taskscited by 3
  22. 2024TutteNet: Injective 3D Deformations by Composition of 2D Mesh Deformationscited by 3
  23. 2022Unsupervised 3D Keypoint Discovery with Multi-View Geometrycited by 2
  24. 2024Controllable Shape Modeling with Neural Generalized Cylindercited by 2
  25. 20243D Registration in 30 Years: A Surveycited by 2
  26. 2024SelfGeo: Self-supervised and Geodesic-consistent Estimation of Keypoints on Deformable Shapescited by 1
  27. 2025CageNet: A Meta-Framework for Learning on Wild Meshescited by 1
  28. 2025ERNet: Efficient Non-Rigid Registration Network for Point Sequencescited by 1
  29. 2026Deep Feature Deformation Weightscited by 1
  30. 2022LatentKeypointGAN: Controlling Images via Latent Keypoints -- Extended Abstractcited by 0
  31. 2025Handle-based Mesh Deformation Guided By Vision Language Modelcited by 0
  32. 2025HPR3D: Hierarchical Proxy Representation for High-Fidelity 3D Reconstruction and Controllable Editingcited by 0
  33. 2025KeyPointDiffuser: Unsupervised 3D Keypoint Learning via Latent Diffusion Modelscited by 0
  34. 2026Adaptive Scaling with Geometric and Visual Continuity of completed 3D objectscited by 0
  35. 2026Category-Level 3D Correspondence in Camera Space via Morphable Object Priorscited by 0
  36. 2026Geometry-Guided Modeling of Foundation Features Enables Generalizable Object Shape Deformation Learningcited by 0
  37. 2026BLARM: Animating 3D Objects from Video via Blending Latent Rigid Motion Primitivescited by 0
  38. 2026OptimusMesh: Compact Autoregressive Mesh Generation from Point Clouds via Sparse Latent Pivotscited by 0