# Equivariant Local Reference Frames for Unsupervised Non-rigid Point Cloud Shape Correspondence

arXiv: https://arxiv.org/abs/2404.00959 · HTML: https://mlsanity.com/paper/2404.00959/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 49 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [Learning from Protein Structure with Geometric Vector Perceptrons](https://mlsanity.com/paper/2009.01411/) (2020)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [Vector Neurons: A General Framework for SO(3)-Equivariant Networks](https://mlsanity.com/paper/2104.12229/) (2021)
- [The Perfect Match: 3D Point Cloud Matching with Smoothed Densities](https://mlsanity.com/paper/1811.06879/) (2018)
- [3D-CODED : 3D Correspondences by Deep Deformation](https://mlsanity.com/paper/1806.05228/) (2018)
- [Deep Geometric Functional Maps: Robust Feature Learning for Shape Correspondence](https://mlsanity.com/paper/2003.14286/) (2020)
- [Deformed Implicit Field: Modeling 3D Shapes with Learned Dense Correspondence](https://mlsanity.com/paper/2011.13650/) (2020)
- [Learning elementary structures for 3D shape generation and matching](https://mlsanity.com/paper/1908.04725/) (2019)
- [Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking](https://mlsanity.com/paper/2111.07786/) (2021)
- [DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data](https://mlsanity.com/paper/1912.04302/) (2019)
- [Smooth Shells: Multi-Scale Shape Registration with Functional Maps](https://mlsanity.com/paper/1905.12512/) (2019)
- [CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphism](https://mlsanity.com/paper/2203.16529/) (2022)
- [Neural Non-Rigid Tracking](https://mlsanity.com/paper/2006.13240/) (2020)
- [CorrNet3D: Unsupervised End-to-end Learning of Dense Correspondence for 3D Point Clouds](https://mlsanity.com/paper/2012.15638/) (2020)
- [GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completion](https://mlsanity.com/paper/2104.05177/) (2021)
- [Unsupervised Deep Multi-Shape Matching](https://mlsanity.com/paper/2207.09610/) (2022)
- [Deep Orientation-Aware Functional Maps: Tackling Symmetry Issues in Shape Matching](https://mlsanity.com/paper/2204.13453/) (2022)
- [Dual Geometric Graph Network (DG2N) -- Iterative network for deformable shape alignment](https://mlsanity.com/paper/2011.14723/) (2020)
- [Equivariant Point Cloud Analysis via Learning Orientations for Message Passing](https://mlsanity.com/paper/2203.14486/) (2022)
- [Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence](https://mlsanity.com/paper/2210.02689/) (2022)
- [Multiway Non-rigid Point Cloud Registration via Learned Functional Map Synchronization](https://mlsanity.com/paper/2111.12878/) (2021)
- [SCOOP: Self-Supervised Correspondence and Optimization-Based Scene Flow](https://mlsanity.com/paper/2211.14020/) (2022)
- [RIGA: Rotation-Invariant and Globally-Aware Descriptors for Point Cloud Registration](https://mlsanity.com/paper/2209.13252/) (2022)
- [Self-Supervised Learning for Multimodal Non-Rigid 3D Shape Matching](https://mlsanity.com/paper/2303.10971/) (2023)

## Built on by (most cited first, 2 of 2)

- [Subequivariant Reinforcement Learning in 3D Multi-Entity Physical Environments](https://mlsanity.com/paper/2407.12505/) (2024)
- [Equivariant Eikonal Neural Networks: Grid-Free, Scalable Travel-Time Prediction on Homogeneous Spaces](https://mlsanity.com/paper/2505.16035/) (2025)

## Abstract (arXiv)

Unsupervised non-rigid point cloud shape correspondence underpins a multitude of 3D vision tasks, yet itself is non-trivial given the exponential complexity stemming from inter-point degree-of-freedom, i.e., pose transformations. Based on the assumption of local rigidity, one solution for reducing complexity is to decompose the overall shape into independent local regions using Local Reference Frames (LRFs) that are invariant to SE(3) transformations. However, the focus solely on local structure neglects global geometric contexts, resulting in less distinctive LRFs that lack crucial semantic information necessary for effective matching. Furthermore, such complexity introduces out-of-distribution geometric contexts during inference, thus complicating generalization. To this end, we introduce 1) EquiShape, a novel structure tailored to learn pair-wise LRFs with global structural cues for both spatial and semantic consistency, and 2) LRF-Refine, an optimization strategy generally applicable to LRF-based methods, aimed at addressing the generalization challenges. Specifically, for EquiShape, we employ cross-talk within separate equivariant graph neural networks (Cross-GVP) to build long-range dependencies to compensate for the lack of semantic information in local structure modeling, deducing pair-wise independent SE(3)-equivariant LRF vectors for each point. For LRF-Refine, the optimization adjusts LRFs within specific contexts and knowledge, enhancing the geometric and semantic generalizability of point features. Our overall framework surpasses the state-of-the-art methods by a large margin on three benchmarks. Code and models will be publicly available.