# GMF: General Multimodal Fusion Framework for Correspondence Outlier Rejection

arXiv: https://arxiv.org/abs/2211.00207 · HTML: https://mlsanity.com/paper/2211.00207/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 1 Nov 2022.

## Builds on (most cited first, 17 of 17)

- [3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions](https://mlsanity.com/paper/1603.08182/) (2016)
- [PREDATOR: Registration of 3D Point Clouds with Low Overlap](https://mlsanity.com/paper/2011.13005/) (2020)
- [TEASER: Fast and Certifiable Point Cloud Registration](https://mlsanity.com/paper/2001.07715/) (2020)
- [Deep Global Registration](https://mlsanity.com/paper/2004.11540/) (2020)
- [Co-Scale Conv-Attentional Image Transformers](https://mlsanity.com/paper/2104.06399/) (2021)
- [CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration](https://mlsanity.com/paper/2110.14076/) (2021)
- [PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency](https://mlsanity.com/paper/2103.05465/) (2021)
- [3DRegNet: A Deep Neural Network for 3D Point Registration](https://mlsanity.com/paper/1904.01701/) (2019)
- [A comprehensive survey on point cloud registration](https://mlsanity.com/paper/2103.02690/) (2021)
- [DeepICP: An End-to-End Deep Neural Network for 3D Point Cloud Registration](https://mlsanity.com/paper/1905.04153/) (2019)
- [Guaranteed Outlier Removal for Point Cloud Registration with Correspondences](https://mlsanity.com/paper/1711.10209/) (2017)
- [SDRSAC: Semidefinite-Based Randomized Approach for Robust Point Cloud Registration without Correspondences](https://mlsanity.com/paper/1904.03483/) (2019)
- [Deep Image: Scaling up Image Recognition](https://mlsanity.com/paper/1501.02876/) (2015)
- [IMFNet: Interpretable Multimodal Fusion for Point Cloud Registration](https://mlsanity.com/paper/2111.09624/) (2021)
- [A Systematic Approach for Cross-source Point Cloud Registration by Preserving Macro and Micro Structures](https://mlsanity.com/paper/1608.05143/) (2016)
- [A coarse-to-fine algorithm for registration in 3D street-view cross-source point clouds](https://mlsanity.com/paper/1610.07324/) (2016)
- [Fast Registration for cross-source point clouds by using weak regional affinity and pixel-wise refinement](https://mlsanity.com/paper/1903.04630/) (2019)

## Built on by (most cited first, 7 of 7)

- [Point-TTA: Test-Time Adaptation for Point Cloud Registration Using Multitask Meta-Auxiliary Learning](https://mlsanity.com/paper/2308.16481/) (2023)
- [An End-to-End Robust Point Cloud Semantic Segmentation Network with Single-Step Conditional Diffusion Models](https://mlsanity.com/paper/2411.16308/) (2024)
- [PointMBF: A Multi-scale Bidirectional Fusion Network for Unsupervised RGB-D Point Cloud Registration](https://mlsanity.com/paper/2308.04782/) (2023)
- [Cross-source Point Cloud Registration: Challenges, Progress and Prospects](https://mlsanity.com/paper/2305.13570/) (2023)
- [DBDNet:Partial-to-Partial Point Cloud Registration with Dual Branches Decoupling](https://mlsanity.com/paper/2310.11733/) (2023)
- [Matching Distance and Geometric Distribution Aided Learning Multiview Point Cloud Registration](https://mlsanity.com/paper/2505.03692/) (2025)
- [LAHNet: Local Attentive Hashing Network for Point Cloud Registration](https://mlsanity.com/paper/2512.00927/) (2025)

## Abstract (arXiv)

Rejecting correspondence outliers enables to boost the correspondence quality, which is a critical step in achieving high point cloud registration accuracy. The current state-of-the-art correspondence outlier rejection methods only utilize the structure features of the correspondences. However, texture information is critical to reject the correspondence outliers in our human vision system. In this paper, we propose General Multimodal Fusion (GMF) to learn to reject the correspondence outliers by leveraging both the structure and texture information. Specifically, two cross-attention-based fusion layers are proposed to fuse the texture information from paired images and structure information from point correspondences. Moreover, we propose a convolutional position encoding layer to enhance the difference between Tokens and enable the encoding feature pay attention to neighbor information. Our position encoding layer will make the cross-attention operation integrate both local and global information. Experiments on multiple datasets(3DMatch, 3DLoMatch, KITTI) and recent state-of-the-art models (3DRegNet, DGR, PointDSC) prove that our GMF achieves wide generalization ability and consistently improves the point cloud registration accuracy. Furthermore, several ablation studies demonstrate the robustness of the proposed GMF on different loss functions, lighting conditions and noises.The code is available at https://github.com/XiaoshuiHuang/GMF.