mlsanityData to 7 Oct 2026

USEEK: Unsupervised SE(3)-Equivariant 3D Keypoints for Generalizable Manipulation

arXiv 2209.13864 ↗First posted 28 Sep 2022cs.RO cs.CVcited by 27

Not on the map. Why?

Abstract (arXiv)

Can a robot manipulate intra-category unseen objects in arbitrary poses with the help of a mere demonstration of grasping pose on a single object instance? In this paper, we try to address this intriguing challenge by using USEEK, an unsupervised SE(3)-equivariant keypoints method that enjoys alignment across instances in a category, to perform generalizable manipulation. USEEK follows a teacher-student structure to decouple the unsupervised keypoint discovery and SE(3)-equivariant keypoint detection. With USEEK in hand, the robot can infer the category-level task-relevant object frames in an efficient and explainable manner, enabling manipulation of any intra-category objects from and to any poses. Through extensive experiments, we demonstrate that the keypoints produced by USEEK possess rich semantics, thus successfully transferring the functional knowledge from the demonstration object to the novel ones. Compared with other object representations for manipulation, USEEK is more adaptive in the face of large intra-category shape variance, more robust with limited demonstrations, and more efficient at inference time.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 29 of its 45 references that are in the map. First posted on arXiv 28 Sep 2022.

Builds on

29 of 45 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
  6. 2018Spherical CNNscited by 452
  7. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 352
  8. 2021Vector Neurons: A General Framework for SO(3)-Equivariant Networkscited by 304
  9. 2021Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulationcited by 204
  10. 2019kPAM: KeyPoint Affordances for Category-Level Robotic Manipulationcited by 152
  11. 2018Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulationcited by 140
  12. 2019Self-Supervised Correspondence in Visuomotor Policy Learningcited by 106
17 more
  1. 2018Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoningcited by 79
  2. 2022You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstrationcited by 79
  3. 20196-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypointscited by 78
  4. 2019USIP: Unsupervised Stable Interest Point Detection from 3D Point Cloudscited by 70
  5. 2020Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learningcited by 69
  6. 2020Learning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth Datacited by 51
  7. 2020KeypointNet: A Large-scale 3D Keypoint Dataset Aggregated from Numerous Human Annotationscited by 50
  8. 2021Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimationcited by 45
  9. 2021Unsupervised Learning of Visual 3D Keypoints for Controlcited by 30
  10. 2020S3K: Self-Supervised Semantic Keypoints for Robotic Manipulation via Multi-View Consistencycited by 28
  11. 2019kPAM-SC: Generalizable Manipulation Planning using KeyPoint Affordance and Shape Completioncited by 27
  12. 2021Skeleton Merger: an Unsupervised Aligned Keypoint Detectorcited by 27
  13. 2018Category-level 6D Object Pose Recovery in Depth Imagescited by 21
  14. 2018Pointwise Rotation-Invariant Network with Adaptive Sampling and 3D Spherical Voxel Convolutioncited by 16
  15. 2020UKPGAN: A General Self-Supervised Keypoint Detectorcited by 15
  16. 2021PRIN/SPRIN: On Extracting Point-wise Rotation Invariant Featurescited by 15
  17. 20133D Interest Point Detection via Discriminative Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 11; 2025: 3; 2026: 4.

27 papers.

Sort:

By area:

  1. 2024EquiBot: SIM(3)-Equivariant Diffusion Policy for Generalizable and Data Efficient Learningcited by 72
  2. 2024Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulationcited by 69
  3. 2023EquivAct: SIM(3)-Equivariant Visuomotor Policies beyond Rigid Object Manipulationcited by 37
  4. 2023Leveraging SE(3) Equivariance for Learning 3D Geometric Shape Assemblycited by 27
  5. 2024ET-SEED: Efficient Trajectory-Level SE(3) Equivariant Diffusion Policycited by 24
  6. 2023ClothesNet: An Information-Rich 3D Garment Model Repository with Simulated Clothes Environmentcited by 22
  7. 2023SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulationcited by 22
  8. 2024UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation via Dense Visual Correspondencecited by 22
  9. 2025SKIL: Semantic Keypoint Imitation Learning for Generalizable Data-efficient Manipulationcited by 21
  10. 2023Banana: Banach Fixed-Point Network for Pointcloud Segmentation with Inter-Part Equivariancecited by 19
  11. 2023EFEM: Equivariant Neural Field Expectation Maximization for 3D Object Segmentation Without Scene Supervisioncited by 18
  12. 2023Leveraging Symmetries in Pick and Placecited by 18
15 more
  1. 2025A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AIcited by 16
  2. 2024Leveraging Locality to Boost Sample Efficiency in Robotic Manipulationcited by 13
  3. 2024RiEMann: Near Real-Time SE(3)-Equivariant Robot Manipulation without Point Cloud Segmentationcited by 12
  4. 2025Two by Two: Learning Multi-Task Pairwise Objects Assembly for Generalizable Robot Manipulationcited by 12
  5. 2023ArrayBot: Reinforcement Learning for Generalizable Distributed Manipulation through Touchcited by 9
  6. 2023SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Datacited by 9
  7. 2024EqvAfford: SE(3) Equivariance for Point-Level Affordance Learningcited by 6
  8. 2024Neural Attention Field: Emerging Point Relevance in 3D Scenes for One-Shot Dexterous Graspingcited by 5
  9. 2024Approximately Piecewise E(3) Equivariant Point Networkscited by 4
  10. 2024Key-Grid: Unsupervised 3D Keypoints Detection using Grid Heatmap Featurescited by 4
  11. 2024SelfGeo: Self-supervised and Geodesic-consistent Estimation of Keypoints on Deformable Shapescited by 1
  12. 2026Interacted Planes Reveal 3D Line Mappingcited by 1
  13. 2026SimpleMatch: A Simple and Strong Baseline for Semantic Correspondencecited by 0
  14. 2026HeteroGenManip: Generalizable Manipulation For Heterogeneous Object Interactionscited by 0
  15. 2026KeyGen: Unsupervised Keypoint based Object-Centric Representations for Category-Level Policy Generalizationcited by 0