# SymbioLCD: Ensemble-Based Loop Closure Detection using CNN-Extracted Objects and Visual Bag-of-Words

arXiv: https://arxiv.org/abs/2110.11491 · HTML: https://mlsanity.com/paper/2110.11491/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 31 references that are in the map.
- First posted on arXiv 21 Oct 2021.

## Builds on (most cited first, 8 of 8)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes](https://mlsanity.com/paper/1806.05620/) (2018)
- [QuadricSLAM: Dual Quadrics from Object Detections as Landmarks in Object-oriented SLAM](https://mlsanity.com/paper/1804.04011/) (2018)
- [On the Performance of ConvNet Features for Place Recognition](https://mlsanity.com/paper/1501.04158/) (2015)
- [Online Visual Place Recognition via Saliency Re-identification](https://mlsanity.com/paper/2007.14549/) (2020)
- [Semantic Mapping with Simultaneous Object Detection and Localization](https://mlsanity.com/paper/1810.11525/) (2018)

## Built on by (most cited first, 2 of 2)

- [Closing the Loop: Graph Networks to Unify Semantic Objects and Visual Features for Multi-object Scenes](https://mlsanity.com/paper/2209.11894/) (2022)
- [Attacking the Loop: Adversarial Attacks on Graph-based Loop Closure Detection](https://mlsanity.com/paper/2312.06991/) (2023)

## Abstract (arXiv)

Loop closure detection is an essential tool of Simultaneous Localization and Mapping (SLAM) to minimize drift in its localization. Many state-of-the-art loop closure detection (LCD) algorithms use visual Bag-of-Words (vBoW), which is robust against partial occlusions in a scene but cannot perceive the semantics or spatial relationships between feature points. CNN object extraction can address those issues, by providing semantic labels and spatial relationships between objects in a scene. Previous work has mainly focused on replacing vBoW with CNN-derived features. In this paper, we propose SymbioLCD, a novel ensemble-based LCD that utilizes both CNN-extracted objects and vBoW features for LCD candidate prediction. When used in tandem, the added elements of object semantics and spatial-awareness create a more robust and symbiotic loop closure detection system. The proposed SymbioLCD uses scale-invariant spatial and semantic matching, Hausdorff distance with temporal constraints, and a Random Forest that utilizes combined information from both CNN-extracted objects and vBoW features for predicting accurate loop closure candidates. Evaluation of the proposed method shows it outperforms other Machine Learning (ML) algorithms - such as SVM, Decision Tree and Neural Network, and demonstrates that there is a strong symbiosis between CNN-extracted object information and vBoW features which assists accurate LCD candidate prediction. Furthermore, it is able to perceive loop closure candidates earlier than state-of-the-art SLAM algorithms, utilizing added spatial and semantic information from CNN-extracted objects.