# HDPV-SLAM: Hybrid Depth-augmented Panoramic Visual SLAM for Mobile Mapping System with Tilted LiDAR and Panoramic Visual Camera arXiv: https://arxiv.org/abs/2301.11823 ยท HTML: https://mlsanity.com/paper/2301.11823/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Builds on 31 of its 65 references that are in the map. - First posted on arXiv 27 Jan 2023. ## Builds on (most cited first, 30 of 31) - [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021) - [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020) - [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016) - [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016) - [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020) - [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016) - [Non-Local Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2007.10042/) (2020) - [CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction](https://mlsanity.com/paper/1704.03489/) (2017) - [PENet: Towards Precise and Efficient Image Guided Depth Completion](https://mlsanity.com/paper/2103.00783/) (2021) - [Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera](https://mlsanity.com/paper/1807.00275/) (2018) - [DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes](https://mlsanity.com/paper/1806.05620/) (2018) - [Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation](https://mlsanity.com/paper/2203.01452/) (2022) - [OpenVSLAM: A Versatile Visual SLAM Framework](https://mlsanity.com/paper/1910.01122/) (2019) - [OmniFusion: 360 Monocular Depth Estimation via Geometry-Aware Fusion](https://mlsanity.com/paper/2203.00838/) (2022) - [Pseudo RGB-D for Self-Improving Monocular SLAM and Depth Prediction](https://mlsanity.com/paper/2004.10681/) (2020) - [ACDNet: Adaptively Combined Dilated Convolution for Monocular Panorama Depth Estimation](https://mlsanity.com/paper/2112.14440/) (2021) - [Eliminating the Blind Spot: Adapting 3D Object Detection and Monocular Depth Estimation to 360deg Panoramic Imagery](https://mlsanity.com/paper/1808.06253/) (2018) - [Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation](https://mlsanity.com/paper/2110.11062/) (2021) - [Unsupervised Monocular Depth Estimation in Highly Complex Environments](https://mlsanity.com/paper/2107.13137/) (2021) - [LIFT-SLAM: a deep-learning feature-based monocular visual SLAM method](https://mlsanity.com/paper/2104.00099/) (2021) - [Improving 360 Monocular Depth Estimation via Non-local Dense Prediction Transformer and Joint Supervised and Self-supervised Learning](https://mlsanity.com/paper/2109.10563/) (2021) - [Omnidirectional DSO: Direct Sparse Odometry with Fisheye Cameras](https://mlsanity.com/paper/1808.02775/) (2018) - [Pose Graph Optimization for Unsupervised Monocular Visual Odometry](https://mlsanity.com/paper/1903.06315/) (2019) - [On Deep Learning Techniques to Boost Monocular Depth Estimation for Autonomous Navigation](https://mlsanity.com/paper/2010.06626/) (2020) - [Keyframe-based monocular SLAM: design, survey, and future directions](https://mlsanity.com/paper/1607.00470/) (2016) - [Deep Depth Estimation from Visual-Inertial SLAM](https://mlsanity.com/paper/2008.00092/) (2020) - [M4Depth: Monocular depth estimation for autonomous vehicles in unseen environments](https://mlsanity.com/paper/2105.09847/) (2021) - [3D-PL: Domain Adaptive Depth Estimation with 3D-aware Pseudo-Labeling](https://mlsanity.com/paper/2209.09231/) (2022) - [DPC-Net: Deep Pose Correction for Visual Localization](https://mlsanity.com/paper/1709.03128/) (2017) - [A Review of Mobile Mapping Systems: From Sensors to Applications](https://mlsanity.com/paper/2205.15865/) (2022) ## Built on by (most cited first, 2 of 2) - [P2U-SLAM: A Monocular Wide-FoV SLAM System Based on Point Uncertainty and Pose Uncertainty](https://mlsanity.com/paper/2409.10143/) (2024) - [A Multi-modal Garden Dataset and Hybrid 3D Dense Reconstruction Framework Based on Panoramic Stereo Images for a Trimming Robot](https://mlsanity.com/paper/2305.06278/) (2023) ## Abstract (arXiv) This paper proposes a novel visual simultaneous localization and mapping (SLAM) system called Hybrid Depth-augmented Panoramic Visual SLAM (HDPV-SLAM), that employs a panoramic camera and a tilted multi-beam LiDAR scanner to generate accurate and metrically-scaled trajectories. RGB-D SLAM was the design basis for HDPV-SLAM, which added depth information to visual features. It aims to solve the two major issues hindering the performance of similar SLAM systems. The first obstacle is the sparseness of LiDAR depth, which makes it difficult to correlate it with the extracted visual features of the RGB image. A deep learning-based depth estimation module for iteratively densifying sparse LiDAR depth was suggested to address this issue. The second issue pertains to the difficulties in depth association caused by a lack of horizontal overlap between the panoramic camera and the tilted LiDAR sensor. To surmount this difficulty, we present a hybrid depth association module that optimally combines depth information estimated by two independent procedures, feature-based triangulation and depth estimation. During a phase of feature tracking, this hybrid depth association module aims to maximize the use of more accurate depth information between the triangulated depth with visual features tracked and the deep learning-based corrected depth. We evaluated the efficacy of HDPV-SLAM using the 18.95 km-long York University and Teledyne Optech (YUTO) MMS dataset. The experimental results demonstrate that the two proposed modules contribute substantially to the performance of HDPV-SLAM, which surpasses that of the state-of-the-art (SOTA) SLAM systems.