# Solving Short-Term Relocalization Problems In Monocular Keyframe Visual SLAM Using Spatial And Semantic Data

arXiv: https://arxiv.org/abs/2407.19518 · HTML: https://mlsanity.com/paper/2407.19518/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 28 Jul 2024.

## Builds on (most cited first, 13 of 13)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [Robot Operating System 2: Design, Architecture, and Uses In The Wild](https://mlsanity.com/paper/2211.07752/) (2022)
- [DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes](https://mlsanity.com/paper/1806.05620/) (2018)
- [CubeSLAM: Monocular 3D Object SLAM](https://mlsanity.com/paper/1806.00557/) (2018)
- [iBoW-LCD: An Appearance-based Loop Closure Detection Approach using Incremental Bags of Binary Words](https://mlsanity.com/paper/1802.05909/) (2018)
- [LIFT-SLAM: a deep-learning feature-based monocular visual SLAM method](https://mlsanity.com/paper/2104.00099/) (2021)
- [Keyframe-based monocular SLAM: design, survey, and future directions](https://mlsanity.com/paper/1607.00470/) (2016)
- [MLPnP - A Real-Time Maximum Likelihood Solution to the Perspective-n-Point Problem](https://mlsanity.com/paper/1607.08112/) (2016)
- [Learning Whole-Image Descriptors for Real-time Loop Detection andKidnap Recovery under Large Viewpoint Difference](https://mlsanity.com/paper/1904.06962/) (2019)
- [DyOb-SLAM : Dynamic Object Tracking SLAM System](https://mlsanity.com/paper/2211.01941/) (2022)

## Abstract (arXiv)

In Monocular Keyframe Visual Simultaneous Localization and Mapping (MKVSLAM) frameworks, when incremental position tracking fails, global pose has to be recovered in a short-time window, also known as short-term relocalization. This capability is crucial for mobile robots to have reliable navigation, build accurate maps, and have precise behaviors around human collaborators. This paper focuses on the development of robust short-term relocalization capabilities for mobile robots using a monocular camera system. A novel multimodal keyframe descriptor is introduced, that contains semantic information of objects detected in the environment and the spatial information of the camera. Using this descriptor, a new Keyframe-based Place Recognition (KPR) method is proposed that is formulated as a multi-stage keyframe filtering algorithm, leading to a new relocalization pipeline for MKVSLAM systems. The proposed approach is evaluated over several indoor GPS denied datasets and demonstrates accurate pose recovery, in comparison to a bag-of-words approach.