# Learning SO(3)-Invariant Semantic Correspondence via Local Shape Transform

arXiv: https://arxiv.org/abs/2404.11156 · HTML: https://mlsanity.com/paper/2404.11156/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 46 references that are in the map.
- First posted on arXiv 17 Apr 2024.

## Builds on (most cited first, 30 of 35)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data](https://mlsanity.com/paper/1908.04616/) (2019)
- [Masked Autoencoders for Point Cloud Self-supervised Learning](https://mlsanity.com/paper/2203.06604/) (2022)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [Vector Neurons: A General Framework for SO(3)-Equivariant Networks](https://mlsanity.com/paper/2104.12229/) (2021)
- [Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completion](https://mlsanity.com/paper/2111.12702/) (2021)
- [3D Point Capsule Networks](https://mlsanity.com/paper/1812.10775/) (2018)
- [Fast Point Transformer](https://mlsanity.com/paper/2112.04702/) (2021)
- [Learning elementary structures for 3D shape generation and matching](https://mlsanity.com/paper/1908.04725/) (2019)
- [Deep Implicit Templates for 3D Shape Representation](https://mlsanity.com/paper/2011.14565/) (2020)
- [Equivariant Point Network for 3D Point Cloud Analysis](https://mlsanity.com/paper/2103.14147/) (2021)
- [CATs: Cost Aggregation Transformers for Visual Correspondence](https://mlsanity.com/paper/2106.02520/) (2021)
- [PointMixer: MLP-Mixer for Point Cloud Understanding](https://mlsanity.com/paper/2111.11187/) (2021)
- [3D Human Mesh Regression with Dense Correspondence](https://mlsanity.com/paper/2006.05734/) (2020)
- [A Rotation-Invariant Framework for Deep Point Cloud Analysis](https://mlsanity.com/paper/2003.07238/) (2020)
- [KeypointNet: A Large-scale 3D Keypoint Dataset Aggregated from Numerous Human Annotations](https://mlsanity.com/paper/2002.12687/) (2020)
- [Dual Geometric Graph Network (DG2N) -- Iterative network for deformable shape alignment](https://mlsanity.com/paper/2011.14723/) (2020)
- [Learning Semantic Correspondence with Sparse Annotations](https://mlsanity.com/paper/2208.06974/) (2022)
- [3D-Rotation-Equivariant Quaternion Neural Networks](https://mlsanity.com/paper/1911.09040/) (2019)
- [Multiway Non-rigid Point Cloud Registration via Learned Functional Map Synchronization](https://mlsanity.com/paper/2111.12878/) (2021)
- [Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences](https://mlsanity.com/paper/2203.04279/) (2022)
- [SRINet: Learning Strictly Rotation-Invariant Representations for Point Cloud Classification and Segmentation](https://mlsanity.com/paper/1911.02163/) (2019)
- [TransforMatcher: Match-to-Match Attention for Semantic Correspondence](https://mlsanity.com/paper/2205.11634/) (2022)
- [Learning Implicit Functions for Topology-Varying Dense 3D Shape Correspondence](https://mlsanity.com/paper/2010.12320/) (2020)
- [Learning 3D Dense Correspondence via Canonical Point Autoencoder](https://mlsanity.com/paper/2107.04867/) (2021)

## Built on by (most cited first, 3 of 3)

- [Learning 3D Scene Analogies with Neural Contextual Scene Maps](https://mlsanity.com/paper/2503.15897/) (2025)
- [RINO: Rotation-Invariant Non-Rigid Correspondences](https://mlsanity.com/paper/2603.27773/) (2026)
- [Similarity-Aware Selective State-Space Modeling for Semantic Correspondence](https://mlsanity.com/paper/2509.24318/) (2025)

## Abstract (arXiv)

Establishing accurate 3D correspondences between shapes stands as a pivotal challenge with profound implications for computer vision and robotics. However, existing self-supervised methods for this problem assume perfect input shape alignment, restricting their real-world applicability. In this work, we introduce a novel self-supervised Rotation-Invariant 3D correspondence learner with Local Shape Transform, dubbed RIST, that learns to establish dense correspondences between shapes even under challenging intra-class variations and arbitrary orientations. Specifically, RIST learns to dynamically formulate an SO(3)-invariant local shape transform for each point, which maps the SO(3)-equivariant global shape descriptor of the input shape to a local shape descriptor. These local shape descriptors are provided as inputs to our decoder to facilitate point cloud self- and cross-reconstruction. Our proposed self-supervised training pipeline encourages semantically corresponding points from different shapes to be mapped to similar local shape descriptors, enabling RIST to establish dense point-wise correspondences. RIST demonstrates state-of-the-art performances on 3D part label transfer and semantic keypoint transfer given arbitrarily rotated point cloud pairs, outperforming existing methods by significant margins.