# A Unified BEV Model for Joint Learning of 3D Local Features and Overlap Estimation

arXiv: https://arxiv.org/abs/2302.14511 · HTML: https://mlsanity.com/paper/2302.14511/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 55 references that are in the map.
- First posted on arXiv 28 Feb 2023.

## Builds on (most cited first, 30 of 36)

- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Circle Loss: A Unified Perspective of Pair Similarity Optimization](https://mlsanity.com/paper/2002.10857/) (2020)
- [3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions](https://mlsanity.com/paper/1603.08182/) (2016)
- [PREDATOR: Registration of 3D Point Clouds with Low Overlap](https://mlsanity.com/paper/2011.13005/) (2020)
- [PointNetLK: Robust & Efficient Point Cloud Registration using PointNet](https://mlsanity.com/paper/1903.05711/) (2019)
- [Deep Global Registration](https://mlsanity.com/paper/2004.11540/) (2020)
- [Geometric Transformer for Fast and Robust Point Cloud Registration](https://mlsanity.com/paper/2202.06688/) (2022)
- [PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition](https://mlsanity.com/paper/1804.03492/) (2018)
- [RPM-Net: Robust Point Matching using Learned Features](https://mlsanity.com/paper/2003.13479/) (2020)
- [D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features](https://mlsanity.com/paper/2003.03164/) (2020)
- [The Perfect Match: 3D Point Cloud Matching with Smoothed Densities](https://mlsanity.com/paper/1811.06879/) (2018)
- [CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration](https://mlsanity.com/paper/2110.14076/) (2021)
- [PRNet: Self-Supervised Learning for Partial-to-Partial Registration](https://mlsanity.com/paper/1910.12240/) (2019)
- [SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration](https://mlsanity.com/paper/2011.12149/) (2020)
- [PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors](https://mlsanity.com/paper/1808.10322/) (2018)
- [MinkLoc3D: Point Cloud Based Large-Scale Place Recognition](https://mlsanity.com/paper/2011.04530/) (2020)
- [Feature-metric Registration: A Fast Semi-supervised Approach for Robust Point Cloud Registration without Correspondences](https://mlsanity.com/paper/2005.01014/) (2020)
- [3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration](https://mlsanity.com/paper/1807.09413/) (2018)
- [Robust Point Cloud Registration Framework Based on Deep Graph Matching](https://mlsanity.com/paper/2103.04256/) (2021)
- [DeepICP: An End-to-End Deep Neural Network for 3D Point Cloud Registration](https://mlsanity.com/paper/1905.04153/) (2019)
- [Scan Context++: Structural Place Recognition Robust to Rotation and Lateral Variations in Urban Environments](https://mlsanity.com/paper/2109.13494/) (2021)
- [OverlapTransformer: An Efficient and Rotation-Invariant Transformer Network for LiDAR-Based Place Recognition](https://mlsanity.com/paper/2203.03397/) (2022)
- [OMNet: Learning Overlapping Mask for Partial-to-Partial Point Cloud Registration](https://mlsanity.com/paper/2103.00937/) (2021)
- [SegMatch: Segment based loop-closure for 3D point clouds](https://mlsanity.com/paper/1609.07720/) (2016)
- [Semantic Graph Based Place Recognition for 3D Point Clouds](https://mlsanity.com/paper/2008.11459/) (2020)
- [SegMap: Segment-based mapping and localization using data-driven descriptors](https://mlsanity.com/paper/1909.12837/) (2019)
- [IMLS-SLAM: scan-to-model matching based on 3D data](https://mlsanity.com/paper/1802.08633/) (2018)
- [LiDAR Iris for Loop-Closure Detection](https://mlsanity.com/paper/1912.03825/) (2019)

## Built on by (most cited first, 3 of 3)

- [RecNet: An Invertible Point Cloud Encoding through Range Image Embeddings for Multi-Robot Map Sharing and Reconstruction](https://mlsanity.com/paper/2402.02192/) (2024)
- [A Coarse-to-Fine Place Recognition Approach using Attention-guided Descriptors and Overlap Estimation](https://mlsanity.com/paper/2303.06881/) (2023)
- [A Strong Baseline for Point Cloud Registration via Direct Superpoints Matching](https://mlsanity.com/paper/2307.01362/) (2023)

## Abstract (arXiv)

Pairwise point cloud registration is a critical task for many applications, which heavily depends on finding correct correspondences from the two point clouds. However, the low overlap between input point clouds causes the registration to fail easily, leading to mistaken overlapping and mismatched correspondences, especially in scenes where non-overlapping regions contain similar structures. In this paper, we present a unified bird's-eye view (BEV) model for jointly learning of 3D local features and overlap estimation to fulfill pairwise registration and loop closure. Feature description is performed by a sparse UNet-like network based on BEV representation, and 3D keypoints are extracted by a detection head for 2D locations, and a regression head for heights. For overlap detection, a cross-attention module is applied for interacting contextual information of input point clouds, followed by a classification head to estimate the overlapping region. We evaluate our unified model extensively on the KITTI dataset and Apollo-SouthBay dataset. The experiments demonstrate that our method significantly outperforms existing methods on overlap estimation, especially in scenes with small overlaps. It also achieves top registration performance on both datasets in terms of translation and rotation errors.