mlsanityData to 14 Oct 2026

ShapeMatcher: Self-Supervised Joint Shape Canonicalization, Segmentation, Retrieval and Deformation

arXiv 2311.11106 ↗First posted 18 Nov 2023cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

In this paper, we present ShapeMatcher, a unified self-supervised learning framework for joint shape canonicalization, segmentation, retrieval and deformation. Given a partially-observed object in an arbitrary pose, we first canonicalize the object by extracting point-wise affine-invariant features, disentangling inherent structure of the object with its pose and size. These learned features are then leveraged to predict semantically consistent part segmentation and corresponding part centers. Next, our lightweight retrieval module aggregates the features within each part as its retrieval token and compare all the tokens with source shapes from a pre-established database to identify the most geometrically similar shape. Finally, we deform the retrieved shape in the deformation module to tightly fit the input object by harnessing part center guided neural cage deformation. The key insight of ShapeMaker is the simultaneous training of the four highly-associated processes: canonicalization, segmentation, retrieval, and deformation, leveraging cross-task consistency losses for mutual supervision. Extensive experiments on synthetic datasets PartNet, ComplementMe, and real-world dataset Scan2CAD demonstrate that ShapeMaker surpasses competitors by a large margin.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 55 of its 74 references that are in the map. First posted on arXiv 18 Nov 2023.

Builds on

55 of 74 references.

References · 55

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  5. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  6. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  7. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  8. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 589
  9. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
  10. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  11. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 488
  12. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
38 more
  1. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 354
  2. 2021Vector Neurons: A General Framework for SO(3)-Equivariant Networkscited by 305
  3. 2019Cormorant: Covariant Molecular Neural Networkscited by 263
  4. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 260
  5. 2017Learning SO(3) Equivariant Representations with Spherical CNNscited by 248
  6. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  7. 2016Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesiscited by 219
  8. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  9. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 172
  10. 2018Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Networkcited by 171
  11. 2019Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Imagescited by 164
  12. 2018Scan2CAD: Learning CAD Model Alignment in RGB-D Scanscited by 142
  13. 2020MeshSDF: Differentiable Iso-Surface Extractioncited by 112
  14. 2019Neural Cages for Detail-Preserving 3D Deformationscited by 106
  15. 2018PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attentioncited by 105
  16. 2021Holistic 3D Scene Understanding from a Single Image with Implicit Representationcited by 94
  17. 2022GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Votingcited by 94
  18. 2021SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimationcited by 88
  19. 2021ROCA: Robust CAD Model Retrieval and Alignment from a Single Imagecited by 67
  20. 20193DN: 3D Deformation Networkcited by 60
  21. 2021KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Controlcited by 51
  22. 2020A Wigner-Eckart Theorem for Group Equivariant Convolution Kernelscited by 46
  23. 2022RBP-Pose: Residual Bounding Box Projection for Category-Level Pose Estimationcited by 43
  24. 2021Joint Learning of 3D Shape Retrieval and Deformationcited by 42
  25. 2022SSP-Pose: Symmetry-Aware Shape Prior Deformation for Direct Category-Level Object Pose Estimationcited by 41
  26. 2019End-to-End CAD Model Retrieval and 9DoF Alignment in 3D Scanscited by 40
  27. 2020Deformation-Aware 3D Model Embedding and Retrievalcited by 30
  28. 2020Canonical Capsules: Self-Supervised Capsules in Canonical Posecited by 30
  29. 2022Masked Autoencoders in 3D Point Cloud Representation Learningcited by 30
  30. 2022PlaneMVS: 3D Plane Reconstruction from Multi-View Stereocited by 29
  31. 2017DeformNet: Free-Form Deformation Network for 3D Shape Reconstruction from a Single Imagecited by 25
  32. 2022MonoGraspNet: 6-DoF Grasping with a Single RGB Imagecited by 24
  33. 2022OPA-3D: Occlusion-Aware Pixel-Wise Aggregation for Monocular 3D Object Detectioncited by 24
  34. 2023SG-Bot: Object Rearrangement via Coarse-to-Fine Robotic Imagination on Scene Graphscited by 24
  35. 2022PatchComplete: Learning Multi-Resolution Patch Priors for 3D Shape Completion on Unseen Categoriescited by 21
  36. 2018Learning Free-Form Deformations for 3D Object Reconstructioncited by 20
  37. 2019Joint Embedding of 3D Scan and CAD Objectscited by 19
  38. 20223D Equivariant Graph Implicit Functionscited by 18

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 4.

8 papers.

  1. 2024UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Imagecited by 12
  2. 2025GIVEPose: Gradual Intra-class Variation Elimination for RGB-based Category-Level Object Pose Estimationcited by 6
  3. 2025MVGSR: Multi-View Consistency Gaussian Splatting for Robust Surface Reconstructioncited by 3
  4. 2025Generative Data Augmentation for Object Point Cloud Segmentationcited by 1
  5. 2026CanoVerse: 3D Object Scalable Canonicalization and Dataset for Generation and Posecited by 1
  6. 2026Geometry-Guided Modeling of Foundation Features Enables Generalizable Object Shape Deformation Learningcited by 0
  7. 2026CANIS: Generation-Assisted 3D Canonicalization via an Image-Semantic Bridgecited by 0
  8. 2026ClothAtlas: Joint Garment Identification and State Estimation under Self-Occlusioncited by 0