# MSSPlace: Multi-Sensor Place Recognition with Visual and Text Semantics

arXiv: https://arxiv.org/abs/2407.15663 · HTML: https://mlsanity.com/paper/2407.15663/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 71 references that are in the map.
- First posted on arXiv 22 Jul 2024.

## Builds on (most cited first, 30 of 45)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [Ferret: Refer and Ground Anything Anywhere at Any Granularity](https://mlsanity.com/paper/2310.07704/) (2023)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [EVA-02: A Visual Representation for Neon Genesis](https://mlsanity.com/paper/2303.11331/) (2023)
- [OneFormer: One Transformer to Rule Universal Image Segmentation](https://mlsanity.com/paper/2211.06220/) (2022)
- [PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition](https://mlsanity.com/paper/1804.03492/) (2018)
- [Rethinking Visual Geo-localization for Large-Scale Applications](https://mlsanity.com/paper/2204.02287/) (2022)
- [AnyLoc: Towards Universal Visual Place Recognition](https://mlsanity.com/paper/2308.00688/) (2023)
- [MixVPR: Feature Mixing for Visual Place Recognition](https://mlsanity.com/paper/2303.02190/) (2023)
- [MinkLoc3D: Point Cloud Based Large-Scale Place Recognition](https://mlsanity.com/paper/2011.04530/) (2020)
- [EigenPlaces: Training Viewpoint Robust Models for Visual Place Recognition](https://mlsanity.com/paper/2308.10832/) (2023)
- [Where is your place, Visual Place Recognition?](https://mlsanity.com/paper/2103.06443/) (2021)
- [OverlapTransformer: An Efficient and Rotation-Invariant Transformer Network for LiDAR-Based Place Recognition](https://mlsanity.com/paper/2203.03397/) (2022)
- [Improving Point Cloud Based Place Recognition with Ranking-based Loss and Large Batch Training](https://mlsanity.com/paper/2203.00972/) (2022)
- [MinkLoc++: Lidar and Monocular Image Fusion for Place Recognition](https://mlsanity.com/paper/2104.05327/) (2021)
- [Text2Pos: Text-to-Point-Cloud Cross-Modal Localization](https://mlsanity.com/paper/2203.15125/) (2022)
- [BEVPlace: Learning LiDAR-based Place Recognition using Bird's Eye View Images](https://mlsanity.com/paper/2302.14325/) (2023)
- [NDT-Transformer: Large-Scale 3D Point Cloud Localisation using the Normal Distribution Transform Representation](https://mlsanity.com/paper/2103.12292/) (2021)
- [SVT-Net: Super Light-Weight Sparse Voxel Transformer for Large Scale Place Recognition](https://mlsanity.com/paper/2105.00149/) (2021)
- [AdaFusion: Visual-LiDAR Fusion with Adaptive Weights for Place Recognition](https://mlsanity.com/paper/2111.11739/) (2021)
- [Single-View Place Recognition under Seasonal Changes](https://mlsanity.com/paper/1808.06516/) (2018)

## Built on by (most cited first, 7 of 7)

- [Ensemble-Based Event Camera Place Recognition Under Varying Illumination](https://mlsanity.com/paper/2509.01968/) (2025)
- [UniMPR: A Unified Framework for Multimodal Place Recognition with Heterogeneous Sensor Configurations](https://mlsanity.com/paper/2512.18279/) (2025)
- [LaVPR: Benchmarking Language and Vision for Place Recognition](https://mlsanity.com/paper/2602.03253/) (2026)
- [M3DMap: Object-aware Multimodal 3D Mapping for Dynamic Environments](https://mlsanity.com/paper/2508.17044/) (2025)
- [PRISM-Loc: a Lightweight Long-range LiDAR Localization in Urban Environments with Topological Maps](https://mlsanity.com/paper/2506.15849/) (2025)
- [Text2Graph VPR: A Text-to-Graph Expert System for Explainable Place Recognition in Changing Environments](https://mlsanity.com/paper/2512.18613/) (2025)
- [MAG-VLAQ: Multi-modal Aerial-Ground Query Aggregation for Cross-View Place Recognition](https://mlsanity.com/paper/2605.09418/) (2026)

## Abstract (arXiv)

Place recognition is a challenging task in computer vision, crucial for enabling autonomous vehicles and robots to navigate previously visited environments. While significant progress has been made in learnable multimodal methods that combine onboard camera images and LiDAR point clouds, the full potential of these methods remains largely unexplored in localization applications. In this paper, we study the impact of leveraging a multi-camera setup and integrating diverse data sources for multimodal place recognition, incorporating explicit visual semantics and text descriptions. Our proposed method named MSSPlace utilizes images from multiple cameras, LiDAR point clouds, semantic segmentation masks, and text annotations to generate comprehensive place descriptors. We employ a late fusion approach to integrate these modalities, providing a unified representation. Through extensive experiments on the Oxford RobotCar and NCLT datasets, we systematically analyze the impact of each data source on the overall quality of place descriptors. Our experiments demonstrate that combining data from multiple sensors significantly improves place recognition model performance compared to single modality approaches and leads to state-of-the-art quality. We also show that separate usage of visual or textual semantics (which are more compact representations of sensory data) can achieve promising results in place recognition. The code for our method is publicly available: https://github.com/alexmelekhin/MSSPlace