DXPR: Depth-Based Vision-LiDAR Cross-Modal Place Recognition Using Vision Foundation Models
Not on the map. Why?
Abstract (arXiv)
We present DXPR, a depth-based cross-modal place recognition (CMPR) framework that uses vision foundation models (VFMs) to match monocular camera queries against a LiDAR map without modality-specific encoders. This enables robots and autonomous vehicles to robustly localize using only cameras within pre-built LiDAR maps, even under severe seasonal, weather, and illumination changes. The key idea is to convert both camera images and LiDAR scans into a unified depth image representation so that a single VFM backbone with an aggregation head can learn modality-invariant global descriptors. To make pairwise metric learning faithful to scene geometry, we introduce a geometry-aware overlap miner: after cross-modal scale alignment of camera and LiDAR depth, we forward-warp measurements between views to compute a pixel-level overlap score. This score relabels ambiguous pairs and adaptively modulates the positive margin in a multi-similarity loss to avoid overfitting on weakly overlapping views. Extensive experiments on KITTI odometry and Boreas demonstrate strong performance and robustness across seasons, weather, and day/night. On KITTI, DXPR achieves near-perfect Recall@1 on most sequences and outperforms prior CMPR baselines. On Boreas, DXPR achieves intra-sequence performance on par with a strong single-modal baseline (DINOv2-SALAD), while showing clear improvements in the more challenging inter-sequence setting. Compared with RangeBEV, our method consistently performs better in both intra- and inter-sequence evaluations, demonstrating robustness under diverse seasonal and illumination changes.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 27 references that are in the map. First posted on arXiv 8 Sep 2026.
Builds on
18 of 27 references.
References · 18
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
- 2019Multi-Similarity Loss with General Pair Weighting for Deep Metric Learningcited by 368
- 2023Optimal Transport Aggregation for Visual Place Recognitioncited by 151
- 2014Learning Rich Features from RGB-D Images for Object Detection and Segmentationcited by 124
- 2022Boreas: A Multi-Season Autonomous Driving Datasetcited by 75
- 2022Patchwork++: Fast and Robust Ground Segmentation Solving Partial Under-Segmentation Using 3D Point Cloudcited by 45
- 2023Data-efficient Large Scale Place Recognition with Graded Similarity Supervisioncited by 43
- 2019Global visual localization in LiDAR-maps through shared 2D-3D embedding spacecited by 28
- 2023HeLiPR: Heterogeneous LiDAR Dataset for inter-LiDAR Place Recognition under Spatiotemporal Variationscited by 26
- 2023LIP-Loc: LiDAR Image Pretraining for Cross-Modal Localizationcited by 26
- 2023(LC)2: LiDAR-Camera Loop Constraints For Cross-Modal Place Recognitioncited by 16
6 more
- 2023I2P-Rec: Recognizing Images on Large-scale Point Cloud Maps through Bird's Eye View Projectionscited by 15
- 2024EffoVPR: Effective Foundation Model Utilization for Visual Place Recognitioncited by 15
- 2025ImLPR: Image-based LiDAR Place Recognition using Vision Foundation Modelscited by 9
- 2024Monocular Visual Place Recognition in LiDAR Maps via Cross-Modal State Space Model and Multi-View Matchingcited by 4
- 2025DOC-Depth: A novel approach for dense depth ground truth generationcited by 3
- 2025Range and Bird's Eye View Fused Cross-Modal Visual Place Recognitioncited by 2
Built on by
Not cited yet.