# SNAKE: Shape-aware Neural 3D Keypoint Field

arXiv: https://arxiv.org/abs/2206.01724 · HTML: https://mlsanity.com/paper/2206.01724/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 53 references that are in the map.
- First posted on arXiv 3 Jun 2022.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions](https://mlsanity.com/paper/1603.08182/) (2016)
- [D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features](https://mlsanity.com/paper/2003.03164/) (2020)
- [Hierarchical Neural Architecture Search for Deep Stereo Matching](https://mlsanity.com/paper/2010.13501/) (2020)
- [Ditto: Building Digital Twins of Articulated Objects from Interaction](https://mlsanity.com/paper/2202.08227/) (2022)
- [Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representations](https://mlsanity.com/paper/2104.01542/) (2021)
- [Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning](https://mlsanity.com/paper/1807.03146/) (2018)
- [USIP: Unsupervised Stable Interest Point Detection from 3D Point Clouds](https://mlsanity.com/paper/1904.00229/) (2019)
- [Fully Convolutional Mesh Autoencoder using Efficient Spatially Varying Kernels](https://mlsanity.com/paper/2006.04325/) (2020)
- [KeypointNet: A Large-scale 3D Keypoint Dataset Aggregated from Numerous Human Annotations](https://mlsanity.com/paper/2002.12687/) (2020)
- [Unsupervised Learning of Category-Specific Symmetric 3D Keypoints from Point Sets](https://mlsanity.com/paper/2003.07619/) (2020)
- [HOPC: Histogram of Oriented Principal Components of 3D Pointclouds for Action Recognition](https://mlsanity.com/paper/1408.3809/) (2014)
- [UKPGAN: A General Self-Supervised Keypoint Detector](https://mlsanity.com/paper/2011.11974/) (2020)
- [Semi-supervised Implicit Scene Completion from Sparse LiDAR](https://mlsanity.com/paper/2111.14798/) (2021)
- [Sim2Real Object-Centric Keypoint Detection and Description](https://mlsanity.com/paper/2202.00448/) (2022)
- [Transferable End-to-end Room Layout Estimation via Implicit Encoding](https://mlsanity.com/paper/2112.11340/) (2021)

## Built on by (most cited first, 10 of 10)

- [SCOOP: Self-Supervised Correspondence and Optimization-Based Scene Flow](https://mlsanity.com/paper/2211.14020/) (2022)
- [BeyondPixels: A Comprehensive Review of the Evolution of Neural Radiance Fields](https://mlsanity.com/paper/2306.03000/) (2023)
- [3D Implicit Transporter for Temporally Consistent Keypoint Discovery](https://mlsanity.com/paper/2309.05098/) (2023)
- [GaussianArt: Unified Modeling of Geometry and Motion for Articulated Objects](https://mlsanity.com/paper/2508.14891/) (2025)
- [FastMAC: Stochastic Spectral Sampling of Correspondence Graph](https://mlsanity.com/paper/2403.08770/) (2024)
- [PartRM: Modeling Part-Level Dynamics with Large Cross-State Reconstruction Model](https://mlsanity.com/paper/2503.19913/) (2025)
- [SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data](https://mlsanity.com/paper/2308.05410/) (2023)
- [ESCAPE: Equivariant Shape Completion via Anchor Point Encoding](https://mlsanity.com/paper/2412.00952/) (2024)
- [SelfGeo: Self-supervised and Geodesic-consistent Estimation of Keypoints on Deformable Shapes](https://mlsanity.com/paper/2408.02291/) (2024)
- [Dual-frame Fluid Motion Estimation with Test-time Optimization and Zero-divergence Loss](https://mlsanity.com/paper/2410.11934/) (2024)

## Abstract (arXiv)

Detecting 3D keypoints from point clouds is important for shape reconstruction, while this work investigates the dual question: can shape reconstruction benefit 3D keypoint detection? Existing methods either seek salient features according to statistics of different orders or learn to predict keypoints that are invariant to transformation. Nevertheless, the idea of incorporating shape reconstruction into 3D keypoint detection is under-explored. We argue that this is restricted by former problem formulations. To this end, a novel unsupervised paradigm named SNAKE is proposed, which is short for shape-aware neural 3D keypoint field. Similar to recent coordinate-based radiance or distance field, our network takes 3D coordinates as inputs and predicts implicit shape indicators and keypoint saliency simultaneously, thus naturally entangling 3D keypoint detection and shape reconstruction. We achieve superior performance on various public benchmarks, including standalone object datasets ModelNet40, KeypointNet, SMPL meshes and scene-level datasets 3DMatch and Redwood. Intrinsic shape awareness brings several advantages as follows. (1) SNAKE generates 3D keypoints consistent with human semantic annotation, even without such supervision. (2) SNAKE outperforms counterparts in terms of repeatability, especially when the input point clouds are down-sampled. (3) the generated keypoints allow accurate geometric registration, notably in a zero-shot setting. Codes are available at https://github.com/zhongcl-thu/SNAKE