# Rotation-Invariant Transformer for Point Cloud Matching

arXiv: https://arxiv.org/abs/2303.08231 · HTML: https://mlsanity.com/paper/2303.08231/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 62 references that are in the map.
- First posted on arXiv 14 Mar 2023.

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [Circle Loss: A Unified Perspective of Pair Similarity Optimization](https://mlsanity.com/paper/2002.10857/) (2020)
- [3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions](https://mlsanity.com/paper/1603.08182/) (2016)
- [PREDATOR: Registration of 3D Point Clouds with Low Overlap](https://mlsanity.com/paper/2011.13005/) (2020)
- [Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts](https://mlsanity.com/paper/2012.09165/) (2020)
- [Geometric Transformer for Fast and Robust Point Cloud Registration](https://mlsanity.com/paper/2202.06688/) (2022)
- [PPFNet: Global Context Aware Local Features for Robust 3D Point Matching](https://mlsanity.com/paper/1802.02669/) (2018)
- [D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features](https://mlsanity.com/paper/2003.03164/) (2020)
- [Neighbourhood Consensus Networks](https://mlsanity.com/paper/1810.10510/) (2018)
- [The Perfect Match: 3D Point Cloud Matching with Smoothed Densities](https://mlsanity.com/paper/1811.06879/) (2018)
- [CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration](https://mlsanity.com/paper/2110.14076/) (2021)
- [SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration](https://mlsanity.com/paper/2011.12149/) (2020)
- [PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors](https://mlsanity.com/paper/1808.10322/) (2018)
- [FLOT: Scene Flow on Point Clouds Guided by Optimal Transport](https://mlsanity.com/paper/2007.11142/) (2020)
- [4DComplete: Non-Rigid Motion Estimation Beyond the Observable Surface](https://mlsanity.com/paper/2105.01905/) (2021)
- [REGTR: End-to-end Point Cloud Correspondences with Transformers](https://mlsanity.com/paper/2203.14517/) (2022)
- [DeepGMR: Learning Latent Gaussian Mixture Models for Registration](https://mlsanity.com/paper/2008.09088/) (2020)
- [Lepard: Learning partial point cloud matching in rigid and deformable scenes](https://mlsanity.com/paper/2111.12591/) (2021)
- [You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors](https://mlsanity.com/paper/2109.00182/) (2021)

## Built on by (most cited first, 30 of 69)

- [Deformable 3D Gaussian Splatting for Animatable Human Avatars](https://mlsanity.com/paper/2312.15059/) (2023)
- [MatchU: Matching Unseen Objects for 6D Pose Estimation from RGB-D Images](https://mlsanity.com/paper/2403.01517/) (2024)
- [NICP: Neural ICP for 3D Human Registration at Scale](https://mlsanity.com/paper/2312.14024/) (2023)
- [GCE-Pose: Global Context Enhancement for Category-level Object Pose Estimation](https://mlsanity.com/paper/2502.04293/) (2025)
- [Scalable 3D Registration via Truncated Entry-wise Absolute Residuals](https://mlsanity.com/paper/2404.00915/) (2024)
- [Deep Graph-based Spatial Consistency for Robust Non-rigid Point Cloud Registration](https://mlsanity.com/paper/2303.09950/) (2023)
- [Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments](https://mlsanity.com/paper/2312.09138/) (2023)
- [Crystalformer: Infinitely Connected Attention for Periodic Structure Encoding](https://mlsanity.com/paper/2403.11686/) (2024)
- [GMSF: Global Matching Scene Flow](https://mlsanity.com/paper/2305.17432/) (2023)
- [Accurate and efficient zero-shot 6D pose estimation with frozen foundation models](https://mlsanity.com/paper/2506.09784/) (2025)
- [TurboReg: TurboClique for Robust and Efficient Point Cloud Registration](https://mlsanity.com/paper/2507.01439/) (2025)
- [CoFiI2P: Coarse-to-Fine Correspondences for Image-to-Point Cloud Registration](https://mlsanity.com/paper/2309.14660/) (2023)
- [Non-Rigid Shape Registration via Deep Functional Maps Prior](https://mlsanity.com/paper/2311.04494/) (2023)
- [SRIF: Semantic Shape Registration Empowered by Diffusion-based Image Morphing and Flow Estimation](https://mlsanity.com/paper/2409.11682/) (2024)
- [UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Image](https://mlsanity.com/paper/2411.16106/) (2024)
- [Beyond Complete Shapes: A Benchmark for Quantitative Evaluation of 3D Shape Surface Matching Algorithms](https://mlsanity.com/paper/2411.03511/) (2024)
- [Register Any Point: Scaling 3D Point Cloud Registration by Flow Matching](https://mlsanity.com/paper/2512.01850/) (2025)
- [CMHANet: A Cross-Modal Hybrid Attention Network for Point Cloud Registration](https://mlsanity.com/paper/2603.12721/) (2026)
- [ZeroReg: Zero-Shot Point Cloud Registration with Foundation Models](https://mlsanity.com/paper/2312.03032/) (2023)
- [Physics-Encoded Graph Neural Networks for Deformation Prediction under Contact](https://mlsanity.com/paper/2402.03466/) (2024)
- [PointDifformer: Robust Point Cloud Registration With Neural Diffusion and Transformer](https://mlsanity.com/paper/2404.14034/) (2024)
- [EM-MIAs: Enhancing Membership Inference Attacks in Large Language Models through Ensemble Modeling](https://mlsanity.com/paper/2412.17249/) (2024)
- [Progressive Correspondence Regenerator for Robust 3D Registration](https://mlsanity.com/paper/2502.02163/) (2025)
- [Unlocking Generalization Power in LiDAR Point Cloud Registration](https://mlsanity.com/paper/2503.10149/) (2025)
- [Detecting Generated Images by Fitting Natural Image Distributions](https://mlsanity.com/paper/2511.01293/) (2025)
- [IGASA: Integrated Geometry-Aware and Skip-Attention Modules for Enhanced Point Cloud Registration](https://mlsanity.com/paper/2603.12719/) (2026)
- [Adaptive Multi-step Refinement Network for Robust Point Cloud Registration](https://mlsanity.com/paper/2312.03053/) (2023)
- [Open-Pose 3D Zero-Shot Learning: Benchmark and Challenges](https://mlsanity.com/paper/2312.07039/) (2023)
- [MaskLRF: Self-supervised Pretraining via Masked Autoencoding of Local Reference Frames for Rotation-invariant 3D Point Set Analysis](https://mlsanity.com/paper/2403.00206/) (2024)
- [SE(3)-bi-equivariant Transformers for Point Cloud Assembly](https://mlsanity.com/paper/2407.09167/) (2024)

## Abstract (arXiv)

The intrinsic rotation invariance lies at the core of matching point clouds with handcrafted descriptors. However, it is widely despised by recent deep matchers that obtain the rotation invariance extrinsically via data augmentation. As the finite number of augmented rotations can never span the continuous SO(3) space, these methods usually show instability when facing rotations that are rarely seen. To this end, we introduce RoITr, a Rotation-Invariant Transformer to cope with the pose variations in the point cloud matching task. We contribute both on the local and global levels. Starting from the local level, we introduce an attention mechanism embedded with Point Pair Feature (PPF)-based coordinates to describe the pose-invariant geometry, upon which a novel attention-based encoder-decoder architecture is constructed. We further propose a global transformer with rotation-invariant cross-frame spatial awareness learned by the self-attention mechanism, which significantly improves the feature distinctiveness and makes the model robust with respect to the low overlap. Experiments are conducted on both the rigid and non-rigid public benchmarks, where RoITr outperforms all the state-of-the-art models by a considerable margin in the low-overlapping scenarios. Especially when the rotations are enlarged on the challenging 3DLoMatch benchmark, RoITr surpasses the existing methods by at least 13 and 5 percentage points in terms of Inlier Ratio and Registration Recall, respectively.