# Deep Depth Estimation from Visual-Inertial SLAM

arXiv: https://arxiv.org/abs/2008.00092 · HTML: https://mlsanity.com/paper/2008.00092/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2020.

## Built on by (most cited first, 13 of 13)

- [Unsupervised Depth Completion with Calibrated Backprojection Layers](https://mlsanity.com/paper/2108.10531/) (2021)
- [Learning Topology from Synthetic Data for Unsupervised Depth Completion](https://mlsanity.com/paper/2106.02994/) (2021)
- [An Adaptive Framework for Learning Unsupervised Depth Completion](https://mlsanity.com/paper/2106.03010/) (2021)
- [Bayesian Deep Basis Fitting for Depth Completion with Uncertainty](https://mlsanity.com/paper/2103.15254/) (2021)
- [RadarCam-Depth: Radar-Camera Fusion for Depth Estimation with Learned Metric Scale](https://mlsanity.com/paper/2401.04325/) (2024)
- [Monocular Visual-Inertial Depth Estimation](https://mlsanity.com/paper/2303.12134/) (2023)
- [Enabling Visual Recognition at Radio Frequency](https://mlsanity.com/paper/2405.19516/) (2024)
- [Struct-MDC: Mesh-Refined Unsupervised Depth Completion Leveraging Structural Regularities from Visual SLAM](https://mlsanity.com/paper/2204.13877/) (2022)
- [Egocentric Scene Understanding via Multimodal Spatial Rectifier](https://mlsanity.com/paper/2207.07077/) (2022)
- [SelfTune: Metrically Scaled Monocular Depth Estimation through Self-Supervised Learning](https://mlsanity.com/paper/2203.05332/) (2022)
- [Sparse SPN: Depth Completion from Sparse Keypoints](https://mlsanity.com/paper/2212.00987/) (2022)
- [HDPV-SLAM: Hybrid Depth-augmented Panoramic Visual SLAM for Mobile Mapping System with Tilted LiDAR and Panoramic Visual Camera](https://mlsanity.com/paper/2301.11823/) (2023)
- [RaCalNet: Radar Calibration Network for Sparse-Supervised Metric Depth Estimation](https://mlsanity.com/paper/2506.15560/) (2025)

## Abstract (arXiv)

This paper addresses the problem of learning to complete a scene's depth from sparse depth points and images of indoor scenes. Specifically, we study the case in which the sparse depth is computed from a visual-inertial simultaneous localization and mapping (VI-SLAM) system. The resulting point cloud has low density, it is noisy, and has non-uniform spatial distribution, as compared to the input from active depth sensors, e.g., LiDAR or Kinect. Since the VI-SLAM produces point clouds only over textured areas, we compensate for the missing depth of the low-texture surfaces by leveraging their planar structures and their surface normals which is an important intermediate representation. The pre-trained surface normal network, however, suffers from large performance degradation when there is a significant difference in the viewing direction (especially the roll angle) of the test image as compared to the trained ones. To address this limitation, we use the available gravity estimate from the VI-SLAM to warp the input image to the orientation prevailing in the training dataset. This results in a significant performance gain for the surface normal estimate, and thus the dense depth estimates. Finally, we show that our method outperforms other state-of-the-art approaches both on training (ScanNet and NYUv2) and testing (collected with Azure Kinect) datasets.