# S3LAM: Structured Scene SLAM

arXiv: https://arxiv.org/abs/2109.07339 · HTML: https://mlsanity.com/paper/2109.07339/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 57 references that are in the map.
- First posted on arXiv 15 Sep 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age](https://mlsanity.com/paper/1606.05830/) (2016)
- [BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth](https://mlsanity.com/paper/1703.10896/) (2017)
- [Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping](https://mlsanity.com/paper/1910.02490/) (2019)
- [CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction](https://mlsanity.com/paper/1704.03489/) (2017)
- [SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks](https://mlsanity.com/paper/1609.05130/) (2016)
- [DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes](https://mlsanity.com/paper/1806.05620/) (2018)
- [CubeSLAM: Monocular 3D Object SLAM](https://mlsanity.com/paper/1806.00557/) (2018)
- [MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects](https://mlsanity.com/paper/1804.09194/) (2018)
- [DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments](https://mlsanity.com/paper/1809.08379/) (2018)
- [QuadricSLAM: Dual Quadrics from Object Detections as Landmarks in Object-oriented SLAM](https://mlsanity.com/paper/1804.04011/) (2018)
- [Semantic Visual Localization](https://mlsanity.com/paper/1712.05773/) (2017)
- [DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAM](https://mlsanity.com/paper/2010.07820/) (2020)
- [Meaningful Maps With Object-Oriented Semantic Mapping](https://mlsanity.com/paper/1609.07849/) (2016)
- [Monocular Object and Plane SLAM in Structured Environments](https://mlsanity.com/paper/1809.03415/) (2018)
- [Is This The Right Place? Geometric-Semantic Pose Verification for Indoor Visual Localization](https://mlsanity.com/paper/1908.04598/) (2019)
- [Pop-up SLAM: Semantic Monocular Plane SLAM for Low-texture Environments](https://mlsanity.com/paper/1703.07334/) (2017)
- [Structure Aware SLAM using Quadrics and Planes](https://mlsanity.com/paper/1804.09111/) (2018)
- [Real-Time Monocular Object-Model Aware Sparse SLAM](https://mlsanity.com/paper/1809.09149/) (2018)
- [ClusterVO: Clustering Moving Instances and Estimating Visual Odometry for Self and Surroundings](https://mlsanity.com/paper/2003.12980/) (2020)
- [Real-time Monocular Object SLAM](https://mlsanity.com/paper/1504.02398/) (2015)
- [L6DNet: Light 6 DoF Network for Robust and Precise Object Pose Estimation with Small Datasets](https://mlsanity.com/paper/2002.00911/) (2020)
- [Semi-Dense 3D Semantic Mapping from Monocular SLAM](https://mlsanity.com/paper/1611.04144/) (2016)

## Built on by (most cited first, 5 of 5)

- [PVO: Panoptic Visual Odometry](https://mlsanity.com/paper/2207.01610/) (2022)
- [A Review on Visual-SLAM: Advancements from Geometric Modelling to Learning-based Semantic Scene Understanding](https://mlsanity.com/paper/2209.05222/) (2022)
- [TwistSLAM: Constrained SLAM in Dynamic Environment](https://mlsanity.com/paper/2202.12384/) (2022)
- [Sparse Image based Navigation Architecture to Mitigate the need of precise Localization in Mobile Robots](https://mlsanity.com/paper/2203.15272/) (2022)
- [TwistSLAM++: Fusing multiple modalities for accurate dynamic semantic SLAM](https://mlsanity.com/paper/2209.07888/) (2022)

## Abstract (arXiv)

We propose a new SLAM system that uses the semantic segmentation of objects and structures in the scene. Semantic information is relevant as it contains high level information which may make SLAM more accurate and robust. Our contribution is twofold: i) A new SLAM system based on ORB-SLAM2 that creates a semantic map made of clusters of points corresponding to objects instances and structures in the scene. ii) A modification of the classical Bundle Adjustment formulation to constrain each cluster using geometrical priors, which improves both camera localization and reconstruction and enables a better understanding of the scene. We evaluate our approach on sequences from several public datasets and show that it improves camera pose estimation with respect to state of the art.