# Learning 3D Dense Correspondence via Canonical Point Autoencoder

arXiv: https://arxiv.org/abs/2107.04867 · HTML: https://mlsanity.com/paper/2107.04867/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 48 references that are in the map.
- First posted on arXiv 10 Jul 2021.

## Builds on (most cited first, 30 of 34)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [Learning Category-Specific Mesh Reconstruction from Image Collections](https://mlsanity.com/paper/1803.07549/) (2018)
- [Learning Shape Templates with Structured Implicit Functions](https://mlsanity.com/paper/1904.06447/) (2019)
- [Learning Shape Abstractions by Assembling Volumetric Primitives](https://mlsanity.com/paper/1612.00404/) (2016)
- [CvxNet: Learnable Convex Decomposition](https://mlsanity.com/paper/1909.05736/) (2019)
- [The Perfect Match: 3D Point Cloud Matching with Smoothed Densities](https://mlsanity.com/paper/1811.06879/) (2018)
- [3D-CODED : 3D Correspondences by Deep Deformation](https://mlsanity.com/paper/1806.05228/) (2018)
- [Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction](https://mlsanity.com/paper/2007.11432/) (2020)
- [Self-supervised Single-view 3D Reconstruction via Semantic Consistency](https://mlsanity.com/paper/2003.06473/) (2020)
- [Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids](https://mlsanity.com/paper/1904.09970/) (2019)
- [Learning elementary structures for 3D shape generation and matching](https://mlsanity.com/paper/1908.04725/) (2019)
- [ParSeNet: A Parametric Surface Fitting Network for 3D Point Clouds](https://mlsanity.com/paper/2003.12181/) (2020)
- [Learning Dense Correspondence via 3D-guided Cycle Consistency](https://mlsanity.com/paper/1604.05383/) (2016)
- [BAE-NET: Branched Autoencoder for Shape Co-Segmentation](https://mlsanity.com/paper/1903.11228/) (2019)
- [3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks](https://mlsanity.com/paper/1708.01648/) (2017)
- [3D Human Mesh Regression with Dense Correspondence](https://mlsanity.com/paper/2006.05734/) (2020)
- [Canonical Surface Mapping via Geometric Cycle Consistency](https://mlsanity.com/paper/1907.10043/) (2019)
- [KeypointNet: A Large-scale 3D Keypoint Dataset Aggregated from Numerous Human Annotations](https://mlsanity.com/paper/2002.12687/) (2020)
- [Implicit Mesh Reconstruction from Unannotated Image Collections](https://mlsanity.com/paper/2007.08504/) (2020)
- [Unsupervised Learning of Intrinsic Structural Representation Points](https://mlsanity.com/paper/2003.01661/) (2020)
- [Deep Parametric Shape Predictions using Distance Fields](https://mlsanity.com/paper/1904.08921/) (2019)
- [Learning Implicit Functions for Topology-Varying Dense 3D Shape Correspondence](https://mlsanity.com/paper/2010.12320/) (2020)

## Built on by (most cited first, 21 of 21)

- [FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Models](https://mlsanity.com/paper/2303.12786/) (2023)
- [Pyramid Diffusion for Fine 3D Large Scene Generation](https://mlsanity.com/paper/2311.12085/) (2023)
- [Autoregressive 3D Shape Generation via Canonical Mapping](https://mlsanity.com/paper/2204.01955/) (2022)
- [Improving Semantic Correspondence with Viewpoint-Guided Spherical Maps](https://mlsanity.com/paper/2312.13216/) (2023)
- [One-Shot Transfer of Affordance Regions? AffCorrs!](https://mlsanity.com/paper/2209.07147/) (2022)
- [NCP: Neural Correspondence Prior for Effective Unsupervised Shape Matching](https://mlsanity.com/paper/2301.05839/) (2023)
- [Back to 3D: Few-Shot 3D Keypoint Detection with Back-Projected 2D Features](https://mlsanity.com/paper/2311.18113/) (2023)
- [Point2SSM: Learning Morphological Variations of Anatomies from Point Cloud](https://mlsanity.com/paper/2305.14486/) (2023)
- [Zero-shot Pose Transfer for Unrigged Stylized 3D Characters](https://mlsanity.com/paper/2306.00200/) (2023)
- [GenCorres: Consistent Shape Matching via Coupled Implicit-Explicit Shape Generative Models](https://mlsanity.com/paper/2304.10523/) (2023)
- [DV-Matcher: Deformation-based Non-Rigid Point Cloud Matching Guided by Pre-trained Visual Features](https://mlsanity.com/paper/2408.08568/) (2024)
- [TUVF: Learning Generalizable Texture UV Radiance Fields](https://mlsanity.com/paper/2305.03040/) (2023)
- [SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data](https://mlsanity.com/paper/2308.05410/) (2023)
- [Arbitrary Point Cloud Upsampling with Spherical Mixture of Gaussians](https://mlsanity.com/paper/2208.05274/) (2022)
- [Deep Deformable Models: Learning 3D Shape Abstractions with Part Consistency](https://mlsanity.com/paper/2309.01035/) (2023)
- [Point2SSM++: Self-Supervised Learning of Anatomical Shape Models from Point Clouds](https://mlsanity.com/paper/2405.09707/) (2024)
- [Learning SO(3)-Invariant Semantic Correspondence via Local Shape Transform](https://mlsanity.com/paper/2404.11156/) (2024)
- [Mitigating Prior Shape Bias in Point Clouds via Differentiable Center Learning](https://mlsanity.com/paper/2402.02088/) (2024)
- [Map Imagination Like Blind Humans: Group Diffusion Model for Robotic Map Generation](https://mlsanity.com/paper/2412.16908/) (2024)
- [NFR: Neural Feature-Guided Non-Rigid Shape Registration](https://mlsanity.com/paper/2505.22445/) (2025)
- [TRACE: Artifact-Robust Statistical Shape Modeling from Imperfect Surface Scans - A Case Study in Craniosynostosis 3D Photography](https://mlsanity.com/paper/2608.22131/) (2026)

## Abstract (arXiv)

We propose a canonical point autoencoder (CPAE) that predicts dense correspondences between 3D shapes of the same category. The autoencoder performs two key functions: (a) encoding an arbitrarily ordered point cloud to a canonical primitive, e.g., a sphere, and (b) decoding the primitive back to the original input instance shape. As being placed in the bottleneck, this primitive plays a key role to map all the unordered point clouds on the canonical surface and to be reconstructed in an ordered fashion. Once trained, points from different shape instances that are mapped to the same locations on the primitive surface are determined to be a pair of correspondence. Our method does not require any form of annotation or self-supervised part segmentation network and can handle unaligned input point clouds. Experimental results on 3D semantic keypoint transfer and part segmentation transfer show that our model performs favorably against state-of-the-art correspondence learning methods.