# Improved Real-Time Monocular SLAM Using Semantic Segmentation on Selective Frames

arXiv: https://arxiv.org/abs/2105.00114 · HTML: https://mlsanity.com/paper/2105.00114/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 51 references that are in the map.
- First posted on arXiv 30 Apr 2021.

## Builds on (most cited first, 11 of 11)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [Improving Semantic Segmentation via Video Propagation and Label Relaxation](https://mlsanity.com/paper/1812.01593/) (2018)
- [Augmented Reality Meets Computer Vision : Efficient Data Generation for Urban Driving Scenes](https://mlsanity.com/paper/1708.01566/) (2017)
- [CubeSLAM: Monocular 3D Object SLAM](https://mlsanity.com/paper/1806.00557/) (2018)
- [DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments](https://mlsanity.com/paper/1809.08379/) (2018)
- [SDCNet: Video Prediction Using Spatially-Displaced Convolution](https://mlsanity.com/paper/1811.00684/) (2018)
- [Keyframe-based monocular SLAM: design, survey, and future directions](https://mlsanity.com/paper/1607.00470/) (2016)
- [Ground Plane based Absolute Scale Estimation for Monocular Visual Odometry](https://mlsanity.com/paper/1903.00912/) (2019)
- [A Unified Framework for Mutual Improvement of SLAM and Semantic Segmentation](https://mlsanity.com/paper/1812.10016/) (2018)

## Built on by (most cited first, 5 of 5)

- [LF-VISLAM: A SLAM Framework for Large Field-of-View Cameras with Negative Imaging Plane on Mobile Agents](https://mlsanity.com/paper/2209.05167/) (2022)
- [Sparse Image based Navigation Architecture to Mitigate the need of precise Localization in Mobile Robots](https://mlsanity.com/paper/2203.15272/) (2022)
- [DK-SLAM: Monocular Visual SLAM with Deep Keypoint Learning, Tracking and Loop-Closing](https://mlsanity.com/paper/2401.09160/) (2024)
- [P2U-SLAM: A Monocular Wide-FoV SLAM System Based on Point Uncertainty and Pose Uncertainty](https://mlsanity.com/paper/2409.10143/) (2024)
- [Geometry-Constrained Monocular Scale Estimation Using Semantic Segmentation for Dynamic Scenes](https://mlsanity.com/paper/2503.04235/) (2025)

## Abstract (arXiv)

Monocular simultaneous localization and mapping (SLAM) is emerging in advanced driver assistance systems and autonomous driving, because a single camera is cheap and easy to install. Conventional monocular SLAM has two major challenges leading inaccurate localization and mapping. First, it is challenging to estimate scales in localization and mapping. Second, conventional monocular SLAM uses inappropriate mapping factors such as dynamic objects and low-parallax areas in mapping. This paper proposes an improved real-time monocular SLAM that resolves the aforementioned challenges by efficiently using deep learning-based semantic segmentation. To achieve the real-time execution of the proposed method, we apply semantic segmentation only to downsampled keyframes in parallel with mapping processes. In addition, the proposed method corrects scales of camera poses and three-dimensional (3D) points, using estimated ground plane from road-labeled 3D points and the real camera height. The proposed method also removes inappropriate corner features labeled as moving objects and low parallax areas. Experiments with eight video sequences demonstrate that the proposed monocular SLAM system achieves significantly improved and comparable trajectory tracking accuracy, compared to existing state-of-the-art monocular and stereo SLAM systems, respectively. The proposed system can achieve real-time tracking on a standard CPU potentially with a standard GPU support, whereas existing segmentation-aided monocular SLAM does not.