# CoFiI2P: Coarse-to-Fine Correspondences for Image-to-Point Cloud Registration arXiv: https://arxiv.org/abs/2309.14660 ยท HTML: https://mlsanity.com/paper/2309.14660/ ## Key facts - Cited by 12 papers in the map (data through 14 Oct 2026). - Builds on 25 of its 48 references that are in the map. - First posted on arXiv 26 Sep 2023. ## Builds on (most cited first, 25 of 25) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019) - [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019) - [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017) - [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019) - [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021) - [Circle Loss: A Unified Perspective of Pair Similarity Optimization](https://mlsanity.com/paper/2002.10857/) (2020) - [Rethinking and Improving Relative Position Encoding for Vision Transformer](https://mlsanity.com/paper/2107.14222/) (2021) - [Geometric Transformer for Fast and Robust Point Cloud Registration](https://mlsanity.com/paper/2202.06688/) (2022) - [COTR: Correspondence Transformer for Matching Across Images](https://mlsanity.com/paper/2103.14167/) (2021) - [The Perfect Match: 3D Point Cloud Matching with Smoothed Densities](https://mlsanity.com/paper/1811.06879/) (2018) - [2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds](https://mlsanity.com/paper/2207.04397/) (2022) - [CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration](https://mlsanity.com/paper/2110.14076/) (2021) - [PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors](https://mlsanity.com/paper/1808.10322/) (2018) - [Patch2Pix: Epipolar-Guided Pixel-Level Correspondences](https://mlsanity.com/paper/2012.01909/) (2020) - [TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation](https://mlsanity.com/paper/2204.05525/) (2022) - [Rotation-Invariant Transformer for Point Cloud Matching](https://mlsanity.com/paper/2303.08231/) (2023) - [2D3D-MatchNet: Learning to Match Keypoints Across 2D Image and 3D Point Cloud](https://mlsanity.com/paper/1904.09742/) (2019) - [DeepI2P: Image-to-Point Cloud Registration via Deep Classification](https://mlsanity.com/paper/2104.03501/) (2021) - [CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking with Camera-LiDAR Fusion](https://mlsanity.com/paper/2209.02540/) (2022) - [Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments](https://mlsanity.com/paper/2103.01627/) (2021) - [CorrI2P: Deep Image-to-Point Cloud Registration via Dense Correspondence](https://mlsanity.com/paper/2207.05483/) (2022) - [P2-Net: Joint Description and Detection of Local Features for Pixel and Point Matching](https://mlsanity.com/paper/2103.01055/) (2021) ## Built on by (most cited first, 12 of 12) - [Iterative Camera-LiDAR Extrinsic Optimization via Surrogate Diffusion](https://mlsanity.com/paper/2411.10936/) (2024) - [GLASS: Geometry-aware Local Alignment and Structure Synchronization Network for 2D-3D Registration](https://mlsanity.com/paper/2603.26262/) (2026) - [Hg-I2P: Bridging Modalities for Generalizable Image-to-Point-Cloud Registration via Heterogeneous Graphs](https://mlsanity.com/paper/2603.27969/) (2026) - [TFCT-I2P: Three stream fusion network with color aware transformer for image-to-point cloud registration](https://mlsanity.com/paper/2410.00360/) (2024) - [Iterative Camera-LiDAR Extrinsic Optimization via Surrogate Diffusion](https://mlsanity.com/paper/2506.14706/) (2025) - [Native-Domain Cross-Attention for Camera-LiDAR Extrinsic Calibration Under Large Initial Perturbations](https://mlsanity.com/paper/2603.29414/) (2026) - [Image-to-Point Cloud Registration Made Easy with Rectified Flow-based LiDAR Upsampling](https://mlsanity.com/paper/2607.14639/) (2026) - [Cross-Coordinate Correspondence Pruning for Image-to-Point Cloud Registration](https://mlsanity.com/paper/2607.17200/) (2026) - [Ex-Sim(3)-Reg: 2D-3D Correspondence Pruning via Extended Sim(3) Registration](https://mlsanity.com/paper/2608.28096/) (2026) - [DRS-VPT: Directly Relocalizing in a Scan with Vision Point Transformers](https://mlsanity.com/paper/2609.12557/) (2026) - [Mask 2D-3D: Adaptive Dual-Masked Autoencoder Network for Image-to-Point Cloud Registration](https://mlsanity.com/paper/2609.18088/) (2026) - [ZIL: Zero-shot Image-to-LiDAR Registration](https://mlsanity.com/paper/2609.22716/) (2026) ## Abstract (arXiv) Image-to-point cloud (I2P) registration is a fundamental task for robots and autonomous vehicles to achieve cross-modality data fusion and localization. Current I2P registration methods primarily focus on estimating correspondences at the point or pixel level, often neglecting global alignment. As a result, I2P matching can easily converge to a local optimum if it lacks high-level guidance from global constraints. To improve the success rate and general robustness, this paper introduces CoFiI2P, a novel I2P registration network that extracts correspondences in a coarse-to-fine manner. First, the image and point cloud data are processed through a two-stream encoder-decoder network for hierarchical feature extraction. Second, a coarse-to-fine matching module is designed to leverage these features and establish robust feature correspondences. Specifically, In the coarse matching phase, a novel I2P transformer module is employed to capture both homogeneous and heterogeneous global information from the image and point cloud data. This enables the estimation of coarse super-point/super-pixel matching pairs with discriminative descriptors. In the fine matching module, point/pixel pairs are established with the guidance of super-point/super-pixel correspondences. Finally, based on matching pairs, the transform matrix is estimated with the EPnP-RANSAC algorithm. Experiments conducted on the KITTI Odometry dataset demonstrate that CoFiI2P achieves impressive results, with a relative rotation error (RRE) of 1.14 degrees and a relative translation error (RTE) of 0.29 meters, while maintaining real-time speed.Additional experiments on the Nuscenes datasets confirm our method's generalizability. The project page is available at urlhttps://whu-usi3dv.github.io/CoFiI2P.