# Loop Closure Detection Based on Object-level Spatial Layout and Semantic Consistency

arXiv: https://arxiv.org/abs/2304.05146 · HTML: https://mlsanity.com/paper/2304.05146/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 11 Apr 2023.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [FastReID: A Pytorch Toolbox for General Instance Re-identification](https://mlsanity.com/paper/2006.02631/) (2020)
- [CubeSLAM: Monocular 3D Object SLAM](https://mlsanity.com/paper/1806.00557/) (2018)
- [Fusion++: Volumetric Object-Level SLAM](https://mlsanity.com/paper/1808.08378/) (2018)
- [Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery](https://mlsanity.com/paper/1903.00268/) (2019)
- [Ground-aware Monocular 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/2102.00690/) (2021)
- [A General Optimization-based Framework for Local Odometry Estimation with Multiple Sensors](https://mlsanity.com/paper/1901.03638/) (2019)
- [X-View: Graph-Based Semantic Multi-View Localization](https://mlsanity.com/paper/1709.09905/) (2017)
- [EAO-SLAM: Monocular Semi-Dense Object SLAM Based on Ensemble Data Association](https://mlsanity.com/paper/2004.12730/) (2020)
- [SO-SLAM: Semantic Object SLAM with Scale Proportional and Symmetrical Texture Constraints](https://mlsanity.com/paper/2109.04884/) (2021)
- [Semantic Histogram Based Graph Matching for Real-Time Multi-Robot Global Localization in Large Scale Environment](https://mlsanity.com/paper/2010.09297/) (2020)
- [Look No Deeper: Recognizing Places from Opposing Viewpoints under Varying Scene Appearance using Single-View Depth Estimation](https://mlsanity.com/paper/1902.07381/) (2019)
- [SemanticLoop: loop closure with 3D semantic graph matching](https://mlsanity.com/paper/2211.11977/) (2022)
- [OrcVIO: Object residual constrained Visual-Inertial Odometry](https://mlsanity.com/paper/2007.15107/) (2020)
- [AirCode: A Robust Object Encoding Method](https://mlsanity.com/paper/2105.00327/) (2021)
- [Closing the Loop: Graph Networks to Unify Semantic Objects and Visual Features for Multi-object Scenes](https://mlsanity.com/paper/2209.11894/) (2022)

## Built on by (most cited first, 1 of 1)

- [Open-Set Semantic Uncertainty Aware Metric-Semantic Graph Matching](https://mlsanity.com/paper/2409.11555/) (2024)

## Abstract (arXiv)

Visual simultaneous localization and mapping (SLAM) systems face challenges in detecting loop closure under the circumstance of large viewpoint changes. In this paper, we present an object-based loop closure detection method based on the spatial layout and semanic consistency of the 3D scene graph. Firstly, we propose an object-level data association approach based on the semantic information from semantic labels, intersection over union (IoU), object color, and object embedding. Subsequently, multi-view bundle adjustment with the associated objects is utilized to jointly optimize the poses of objects and cameras. We represent the refined objects as a 3D spatial graph with semantics and topology. Then, we propose a graph matching approach to select correspondence objects based on the structure layout and semantic property similarity of vertices' neighbors. Finally, we jointly optimize camera trajectories and object poses in an object-level pose graph optimization, which results in a globally consistent map. Experimental results demonstrate that our proposed data association approach can construct more accurate 3D semantic maps, and our loop closure method is more robust than point-based and object-based methods in circumstances with large viewpoint changes.