# Unifying Local and Global Multimodal Features for Place Recognition in Aliased and Low-Texture Environments

arXiv: https://arxiv.org/abs/2403.13395 · HTML: https://mlsanity.com/paper/2403.13395/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 54 references that are in the map.
- First posted on arXiv 20 Mar 2024.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition](https://mlsanity.com/paper/2103.01486/) (2021)
- [Visual Attention Network](https://mlsanity.com/paper/2202.09741/) (2022)
- [PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition](https://mlsanity.com/paper/1804.03492/) (2018)
- [The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Dataset](https://mlsanity.com/paper/1909.01300/) (2019)
- [TransVPR: Transformer-based place recognition with multi-level attention aggregation](https://mlsanity.com/paper/2201.02001/) (2022)
- [The TUM VI Benchmark for Evaluating Visual-Inertial Odometry](https://mlsanity.com/paper/1804.06120/) (2018)
- [Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling](https://mlsanity.com/paper/2301.03580/) (2023)
- [Where is your place, Visual Place Recognition?](https://mlsanity.com/paper/2103.06443/) (2021)
- [OverlapTransformer: An Efficient and Rotation-Invariant Transformer Network for LiDAR-Based Place Recognition](https://mlsanity.com/paper/2203.03397/) (2022)
- [Occupancy-MAE: Self-supervised Pre-training Large-scale LiDAR Point Clouds with Masked Occupancy Autoencoders](https://mlsanity.com/paper/2206.09900/) (2022)
- [4Seasons: A Cross-Season Dataset for Multi-Weather SLAM in Autonomous Driving](https://mlsanity.com/paper/2009.06364/) (2020)
- [MinkLoc++: Lidar and Monocular Image Fusion for Place Recognition](https://mlsanity.com/paper/2104.05327/) (2021)
- [Masked Autoencoder for Self-Supervised Pre-training on Lidar Point Clouds](https://mlsanity.com/paper/2207.00531/) (2022)
- [AdaFusion: Visual-LiDAR Fusion with Adaptive Weights for Place Recognition](https://mlsanity.com/paper/2111.11739/) (2021)
- [Learning Super-Features for Image Retrieval](https://mlsanity.com/paper/2201.13182/) (2022)
- [Visual Place Recognition: A Tutorial](https://mlsanity.com/paper/2303.03281/) (2023)
- [TransLoc3D : Point Cloud based Large-scale Place Recognition using Adaptive Receptive Fields](https://mlsanity.com/paper/2105.11605/) (2021)
- [Attention-based Pyramid Aggregation Network for Visual Place Recognition](https://mlsanity.com/paper/1808.00288/) (2018)
- [VIZARD: Reliable Visual Localization for Autonomous Vehicles in Urban Outdoor Environments](https://mlsanity.com/paper/1902.04343/) (2019)
- [Challenges of SLAM in extremely unstructured environments: the DLR Planetary Stereo, Solid-State LiDAR, Inertial Dataset](https://mlsanity.com/paper/2207.06815/) (2022)
- [SLAM in the Field: An Evaluation of Monocular Mapping and Localization on Challenging Dynamic Agricultural Environment](https://mlsanity.com/paper/2011.01122/) (2020)

## Built on by (most cited first, 7 of 7)

- [UAVScenes: A Multi-Modal Dataset for UAVs](https://mlsanity.com/paper/2507.22412/) (2025)
- [MSSPlace: Multi-Sensor Place Recognition with Visual and Text Semantics](https://mlsanity.com/paper/2407.15663/) (2024)
- [PRFusion: Toward Effective and Robust Multi-Modal Place Recognition with Image and Point Cloud Fusion](https://mlsanity.com/paper/2410.04939/) (2024)
- [Multi-modal Loop Closure Detection with Foundation Models in Severely Unstructured Environments](https://mlsanity.com/paper/2511.05404/) (2025)
- [M3DMap: Object-aware Multimodal 3D Mapping for Dynamic Environments](https://mlsanity.com/paper/2508.17044/) (2025)
- [MAG-VLAQ: Multi-modal Aerial-Ground Query Aggregation for Cross-View Place Recognition](https://mlsanity.com/paper/2605.09418/) (2026)
- [M3GA-Wild: A Large-Scale Dataset and Benchmark for Multi-Modal Multi-session Ground-to-Aerial Place Recognition in Forests](https://mlsanity.com/paper/2609.23003/) (2026)

## Abstract (arXiv)

Perceptual aliasing and weak textures pose significant challenges to the task of place recognition, hindering the performance of Simultaneous Localization and Mapping (SLAM) systems. This paper presents a novel model, called UMF (standing for Unifying Local and Global Multimodal Features) that 1) leverages multi-modality by cross-attention blocks between vision and LiDAR features, and 2) includes a re-ranking stage that re-orders based on local feature matching the top-k candidates retrieved using a global representation. Our experiments, particularly on sequences captured on a planetary-analogous environment, show that UMF outperforms significantly previous baselines in those challenging aliased environments. Since our work aims to enhance the reliability of SLAM in all situations, we also explore its performance on the widely used RobotCar dataset, for broader applicability. Code and models are available at https://github.com/DLR-RM/UMF