# CNN-Augmented Visual-Inertial SLAM with Planar Constraints

arXiv: https://arxiv.org/abs/2205.02940 · HTML: https://mlsanity.com/paper/2205.02940/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 34 references that are in the map.
- First posted on arXiv 5 May 2022.

## Builds on (most cited first, 20 of 20)

- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age](https://mlsanity.com/paper/1606.05830/) (2016)
- [D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry](https://mlsanity.com/paper/2003.01060/) (2020)
- [CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction](https://mlsanity.com/paper/1704.03489/) (2017)
- [Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry](https://mlsanity.com/paper/1807.02570/) (2018)
- [Pseudo RGB-D for Self-Improving Monocular SLAM and Depth Prediction](https://mlsanity.com/paper/2004.10681/) (2020)
- [Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling](https://mlsanity.com/paper/2007.10983/) (2020)
- [MonoIndoor: Towards Good Practice of Self-Supervised Monocular Depth Estimation for Indoor Environments](https://mlsanity.com/paper/2107.12429/) (2021)
- [PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2203.12082/) (2022)
- [Pop-up SLAM: Semantic Monocular Plane SLAM for Low-texture Environments](https://mlsanity.com/paper/1703.07334/) (2017)
- [Structure Aware SLAM using Quadrics and Planes](https://mlsanity.com/paper/1804.09111/) (2018)
- [CNN-SVO: Improving the Mapping in Semi-Direct Visual Odometry Using Single-Image Depth Prediction](https://mlsanity.com/paper/1810.01011/) (2018)
- [An Efficient Planar Bundle Adjustment Algorithm](https://mlsanity.com/paper/2006.00187/) (2020)
- [GeoRefine: Self-Supervised Online Depth Refinement for Accurate Dense Mapping](https://mlsanity.com/paper/2205.01656/) (2022)

## Abstract (arXiv)

We present a robust visual-inertial SLAM system that combines the benefits of Convolutional Neural Networks (CNNs) and planar constraints. Our system leverages a CNN to predict the depth map and the corresponding uncertainty map for each image. The CNN depth effectively bootstraps the back-end optimization of SLAM and meanwhile the CNN uncertainty adaptively weighs the contribution of each feature point to the back-end optimization. Given the gravity direction from the inertial sensor, we further present a fast plane detection method that detects horizontal planes via one-point RANSAC and vertical planes via two-point RANSAC. Those stably detected planes are in turn used to regularize the back-end optimization of SLAM. We evaluate our system on a public dataset, ie, EuRoC, and demonstrate improved results over a state-of-the-art SLAM system, ie, ORB-SLAM3.