mlsanityData to 14 Oct 2026

Semantic-Enhanced Cross-Modal Place Recognition for Robust Robot Localization

arXiv 2509.13474 ↗First posted 16 Sep 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Ensuring accurate localization of robots in environments without GPS capability is a challenging task. Visual Place Recognition (VPR) techniques can potentially achieve this goal, but existing RGB-based methods are sensitive to changes in illumination, weather, and other seasonal changes. Existing cross-modal localization methods leverage the geometric properties of RGB images and 3D LiDAR maps to reduce the sensitivity issues highlighted above. Currently, state-of-the-art methods struggle in complex scenes, fine-grained or high-resolution matching, and situations where changes can occur in viewpoint. In this work, we introduce a framework we call Semantic-Enhanced Cross-Modal Place Recognition (SCM-PR) that combines high-level semantics utilizing RGB images for robust localization in LiDAR maps. Our proposed method introduces: a VMamba backbone for feature extraction of RGB images; a Semantic-Aware Feature Fusion (SAFF) module for using both place descriptors and segmentation masks; LiDAR descriptors that incorporate both semantics and geometry; and a cross-modal semantic attention mechanism in NetVLAD to improve matching. Incorporating the semantic information also was instrumental in designing a Multi-View Semantic-Geometric Matching and a Semantic Consistency Loss, both in a contrastive learning framework. Our experimental work on the KITTI and KITTI-360 datasets show that SCM-PR achieves state-of-the-art performance compared to other cross-modal place recognition methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 41 references that are in the map. First posted on arXiv 16 Sep 2025.

Builds on

30 of 41 references.

References · 30

  1. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  2. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
  3. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  4. 2021Muppet: Massive Multi-task Representations with Pre-Finetuningcited by 192
  5. 2022Cross-modal Memory Networks for Radiology Report Generationcited by 185
  6. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 180
  7. 2021Template-Based Named Entity Recognition Using BARTcited by 153
  8. 2024Visual In-Context Learning for Large Vision-Language Modelscited by 128
  9. 2020Visual News: Benchmark and Challenges in News Image Captioningcited by 121
  10. 2022ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehensioncited by 120
  11. 2021Probing Image-Language Transformers for Verb Understandingcited by 109
  12. 2021E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learningcited by 85
18 more
  1. 2022Cross-modal Contrastive Learning for Speech Translationcited by 61
  2. 2021On Pursuit of Designing Multi-modal Transformer for Video Groundingcited by 54
  3. 2025Improving Medical Large Vision-Language Models with Abnormal-Aware Feedbackcited by 42
  4. 2020Meta-Learning for Domain Generalization in Semantic Parsingcited by 41
  5. 2021Multimodal End-to-End Sparse Model for Emotion Recognitioncited by 40
  6. 2021KITTI-CARLA: a KITTI-like dataset generated by CARLA Simulatorcited by 34
  7. 2021Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarizationcited by 32
  8. 2024LAMBDA: A Large Model Based Data Agentcited by 30
  9. 2025Draw ALL Your Imagine: A Holistic Benchmark and Agent Framework for Complex Instruction-based Image Generationcited by 30
  10. 2020N-LTP: An Open-source Neural Language Technology Platform for Chinesecited by 29
  11. 2023Learning Language-guided Adaptive Hyper-modality Representation for Multimodal Sentiment Analysiscited by 29
  12. 2021ITA: Image-Text Alignments for Multi-Modal Named Entity Recognitioncited by 24
  13. 2024A Survey on Large Language Model-based Agents for Statistics and Data Sciencecited by 22
  14. 2022CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detectioncited by 21
  15. 2021Multiplex Graph Neural Network for Extractive Text Summarizationcited by 19
  16. 2024LlamaCare: A Large Medical Language Model for Enhancing Healthcare Knowledge Sharingcited by 4
  17. 2025Adaptive Field Effect Planner for Safe Interactive Autonomous Driving on Curved Roadscited by 2
  18. 2025Data-Driven Evolutionary Game-Based Model Predictive Control for Hybrid Renewable Energy Dispatch in Autonomous Shipscited by 2

Built on by

Not cited yet.