# An Online Semantic Mapping System for Extending and Enhancing Visual SLAM

arXiv: https://arxiv.org/abs/2203.03944 · HTML: https://mlsanity.com/paper/2203.03944/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 44 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 13 of 13)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks](https://mlsanity.com/paper/1609.05130/) (2016)
- [DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes](https://mlsanity.com/paper/1806.05620/) (2018)
- [CubeSLAM: Monocular 3D Object SLAM](https://mlsanity.com/paper/1806.00557/) (2018)
- [Fusion++: Volumetric Object-Level SLAM](https://mlsanity.com/paper/1808.08378/) (2018)
- [DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments](https://mlsanity.com/paper/1809.08379/) (2018)
- [QuadricSLAM: Dual Quadrics from Object Detections as Landmarks in Object-oriented SLAM](https://mlsanity.com/paper/1804.04011/) (2018)
- [Probabilistic Data Association via Mixture Models for Robust Semantic SLAM](https://mlsanity.com/paper/1909.11213/) (2019)
- [Fast and Accurate Semantic Mapping through Geometric-based Incremental Segmentation](https://mlsanity.com/paper/1803.02784/) (2018)
- [Multiple Hypothesis Semantic Mapping for Robust Data Association](https://mlsanity.com/paper/2012.04423/) (2020)
- [Extending Maps with Semantic and Contextual Object Information for Robot Navigation: a Learning-Based Framework using Visual and Depth Cues](https://mlsanity.com/paper/2003.06336/) (2020)

## Built on by (most cited first, 3 of 3)

- [Leveraging Large Language Model-based Room-Object Relationships Knowledge for Enhancing Multimodal-Input Object Goal Navigation](https://mlsanity.com/paper/2403.14163/) (2024)
- [Object-Oriented Material Classification and 3D Clustering for Improved Semantic Perception and Mapping in Mobile Robots](https://mlsanity.com/paper/2407.06077/) (2024)
- [Automating 3D Dataset Generation with Neural Radiance Fields](https://mlsanity.com/paper/2503.15997/) (2025)

## Abstract (arXiv)

We present a real-time semantic mapping approach for mobile vision systems with a 2D to 3D object detection pipeline and rapid data association for generated landmarks. Besides the semantic map enrichment the associated detections are further introduced as semantic constraints into a simultaneous localization and mapping (SLAM) system for pose correction purposes. This way, we are able generate additional meaningful information that allows to achieve higher-level tasks, while simultaneously leveraging the view-invariance of object detections to improve the accuracy and the robustness of the odometry estimation. We propose tracklets of locally associated object observations to handle ambiguous and false predictions and an uncertainty-based greedy association scheme for an accelerated processing time. Our system reaches real-time capabilities with an average iteration duration of 65~ms and is able to improve the pose estimation of a state-of-the-art SLAM by up to 68% on a public dataset. Additionally, we implemented our approach as a modular ROS package that makes it straightforward for integration in arbitrary graph-based SLAM methods.