# Semantic Mapping with Simultaneous Object Detection and Localization

arXiv: https://arxiv.org/abs/1810.11525 · HTML: https://mlsanity.com/paper/1810.11525/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2018.

## Built on by (most cited first, 4 of 4)

- [Dynamic Semantic Occupancy Mapping using 3D Scene Flow and Closed-Form Bayesian Inference](https://mlsanity.com/paper/2108.03180/) (2021)
- [Closing the Loop: Graph Networks to Unify Semantic Objects and Visual Features for Multi-object Scenes](https://mlsanity.com/paper/2209.11894/) (2022)
- [SymbioLCD: Ensemble-Based Loop Closure Detection using CNN-Extracted Objects and Visual Bag-of-Words](https://mlsanity.com/paper/2110.11491/) (2021)
- [Artifacts Mapping: Multi-Modal Semantic Mapping for Object Detection and 3D Localization](https://mlsanity.com/paper/2307.01121/) (2023)

## Abstract (arXiv)

We present a filtering-based method for semantic mapping to simultaneously detect objects and localize their 6 degree-of-freedom pose. For our method, called Contextual Temporal Mapping (or CT-Map), we represent the semantic map as a belief over object classes and poses across an observed scene. Inference for the semantic mapping problem is then modeled in the form of a Conditional Random Field (CRF). CT-Map is a CRF that considers two forms of relationship potentials to account for contextual relations between objects and temporal consistency of object poses, as well as a measurement potential on observations. A particle filtering algorithm is then proposed to perform inference in the CT-Map model. We demonstrate the efficacy of the CT-Map method with a Michigan Progress Fetch robot equipped with a RGB-D sensor. Our results demonstrate that the particle filtering based inference of CT-Map provides improved object detection and pose estimation with respect to baseline methods that treat observations as independent samples of a scene.