# CMRNet: Camera to LiDAR-Map Registration

arXiv: https://arxiv.org/abs/1906.10109 · HTML: https://mlsanity.com/paper/1906.10109/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2019.

## Built on by (most cited first, 25 of 25)

- [Differentiable Registration of Images and LiDAR Point Clouds with VoxelPoint-to-Pixel Matching](https://mlsanity.com/paper/2312.04060/) (2023)
- [LIP-Loc: LiDAR Image Pretraining for Cross-Modal Localization](https://mlsanity.com/paper/2312.16648/) (2023)
- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [(LC)2: LiDAR-Camera Loop Constraints For Cross-Modal Place Recognition](https://mlsanity.com/paper/2304.08660/) (2023)
- [CMRNext: Camera to LiDAR Matching in the Wild for Localization and Extrinsic Calibration](https://mlsanity.com/paper/2402.00129/) (2024)
- [Robust Monocular Localization in Sparse HD Maps Leveraging Multi-Task Uncertainty Estimation](https://mlsanity.com/paper/2110.10563/) (2021)
- [End-to-end 2D-3D Registration between Image and LiDAR Point Cloud for Vehicle Localization](https://mlsanity.com/paper/2306.11346/) (2023)
- [VOLoc: Visual Place Recognition by Querying Compressed Lidar Map](https://mlsanity.com/paper/2402.15961/) (2024)
- [RaLF: Flow-based Global and Metric Radar Localization in LiDAR Maps](https://mlsanity.com/paper/2309.09875/) (2023)
- [VXP: Voxel-Cross-Pixel Large-scale Image-LiDAR Place Recognition](https://mlsanity.com/paper/2403.14594/) (2024)
- [EffLoc: Lightweight Vision Transformer for Efficient 6-DOF Camera Relocalization](https://mlsanity.com/paper/2402.13537/) (2024)
- [CMR-Agent: Learning a Cross-Modal Agent for Iterative Image-to-Point Cloud Registration](https://mlsanity.com/paper/2408.02394/) (2024)
- [Revisiting LiDAR Registration and Reconstruction: A Range Image Perspective](https://mlsanity.com/paper/2112.02779/) (2021)
- [Poses as Queries: Image-to-LiDAR Map Localization with Transformers](https://mlsanity.com/paper/2305.04298/) (2023)
- [SOLVR: Submap Oriented LiDAR-Visual Re-Localisation](https://mlsanity.com/paper/2409.10247/) (2024)
- [What Really Matters for Learning-based LiDAR-Camera Calibration](https://mlsanity.com/paper/2501.16969/) (2025)
- [Semantic Image Alignment for Vehicle Localization](https://mlsanity.com/paper/2110.04162/) (2021)
- [A Survey on Visual Map Localization Using LiDARs and Cameras](https://mlsanity.com/paper/2208.03376/) (2022)
- [Uncertainty-Aware DNN for Multi-Modal Camera Localization](https://mlsanity.com/paper/2211.01234/) (2022)
- [RobustCalib: Robust Lidar-Camera Extrinsic Calibration with Consistency Learning](https://mlsanity.com/paper/2312.01085/) (2023)
- [An experimental evaluation of Siamese Neural Networks for robot localization using omnidirectional imaging in indoor environments](https://mlsanity.com/paper/2407.10536/) (2024)
- [Self-localization on a 3D map by fusing global and local features from a monocular camera](https://mlsanity.com/paper/2510.26170/) (2025)
- [CRISTAL: Real-time Camera Registration in Static LiDAR Scans using Neural Rendering](https://mlsanity.com/paper/2511.16349/) (2025)
- [Joint Multi-Camera LiDAR Extrinsic Calibration via Learned Pairwise Initialization and Geometric Refinement](https://mlsanity.com/paper/2605.31576/) (2026)
- [Ex-Sim(3)-Reg: 2D-3D Correspondence Pruning via Extended Sim(3) Registration](https://mlsanity.com/paper/2608.28096/) (2026)

## Abstract (arXiv)

In this paper we present CMRNet, a realtime approach based on a Convolutional Neural Network to localize an RGB image of a scene in a map built from LiDAR data. Our network is not trained in the working area, i.e. CMRNet does not learn the map. Instead it learns to match an image to the map. We validate our approach on the KITTI dataset, processing each frame independently without any tracking procedure. CMRNet achieves 0.27m and 1.07deg median localization accuracy on the sequence 00 of the odometry dataset, starting from a rough pose estimate displaced up to 3.5m and 17deg. To the best of our knowledge this is the first CNN-based approach that learns to match images from a monocular camera to a given, preexisting 3D LiDAR-map.