# Sparse and noisy LiDAR completion with RGB guidance and uncertainty

arXiv: https://arxiv.org/abs/1902.05356 · HTML: https://mlsanity.com/paper/1902.05356/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2019.

## Built on by (most cited first, 30 of 79)

- [PENet: Towards Precise and Efficient Image Guided Depth Completion](https://mlsanity.com/paper/2103.00783/) (2021)
- [S-NeRF: Neural Radiance Fields for Street Views](https://mlsanity.com/paper/2303.00749/) (2023)
- [CompletionFormer: Depth Completion with Convolutions and Vision Transformers](https://mlsanity.com/paper/2304.13030/) (2023)
- [RigNet: Repetitive Image Guided Network for Depth Completion](https://mlsanity.com/paper/2107.13802/) (2021)
- [Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors](https://mlsanity.com/paper/2503.17316/) (2025)
- [Dynamic Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2202.09769/) (2022)
- [Unsupervised Depth Completion with Calibrated Backprojection Layers](https://mlsanity.com/paper/2108.10531/) (2021)
- [Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects](https://mlsanity.com/paper/2208.03792/) (2022)
- [Tri-Perspective View Decomposition for Geometry-Aware Depth Completion](https://mlsanity.com/paper/2403.15008/) (2024)
- [Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completion](https://mlsanity.com/paper/2103.16690/) (2021)
- [Learning Topology from Synthetic Data for Unsupervised Depth Completion](https://mlsanity.com/paper/2106.02994/) (2021)
- [Deep Depth Completion from Extremely Sparse Data: A Survey](https://mlsanity.com/paper/2205.05335/) (2022)
- [An Adaptive Framework for Learning Unsupervised Depth Completion](https://mlsanity.com/paper/2106.03010/) (2021)
- [S-NeRF++: Autonomous Driving Simulation via Neural Reconstruction and Generation](https://mlsanity.com/paper/2402.02112/) (2024)
- [DesNet: Decomposed Scale-Consistent Network for Unsupervised Depth Completion](https://mlsanity.com/paper/2211.10994/) (2022)
- [BIPS: Bi-modal Indoor Panorama Synthesis via Residual Depth-aided Adversarial Learning](https://mlsanity.com/paper/2112.06179/) (2021)
- [Test-Time Adaptation for Depth Completion](https://mlsanity.com/paper/2402.03312/) (2024)
- [Depth Completion using Plane-Residual Representation](https://mlsanity.com/paper/2104.07350/) (2021)
- [Monitored Distillation for Positive Congruent Depth Completion](https://mlsanity.com/paper/2203.16034/) (2022)
- [Bayesian Deep Basis Fitting for Depth Completion with Uncertainty](https://mlsanity.com/paper/2103.15254/) (2021)
- [Multi-Modal Masked Pre-Training for Monocular Panoramic Depth Completion](https://mlsanity.com/paper/2203.09855/) (2022)
- [SemAttNet: Towards Attention-based Semantic Aware Guided Depth Completion](https://mlsanity.com/paper/2204.13635/) (2022)
- [RadarCam-Depth: Radar-Camera Fusion for Depth Estimation with Learned Metric Scale](https://mlsanity.com/paper/2401.04325/) (2024)
- [Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution](https://mlsanity.com/paper/2306.00386/) (2023)
- [A Surface Geometry Model for LiDAR Depth Completion](https://mlsanity.com/paper/2104.08466/) (2021)
- [RGB-Depth Fusion GAN for Indoor Depth Completion](https://mlsanity.com/paper/2203.10856/) (2022)
- [DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Network](https://mlsanity.com/paper/2108.12655/) (2021)
- [Robust Depth Completion with Uncertainty-Driven Loss Functions](https://mlsanity.com/paper/2112.07895/) (2021)
- [AugUndo: Scaling Up Augmentations for Monocular Depth Completion and Estimation](https://mlsanity.com/paper/2310.09739/) (2023)
- [Bayesian Imitation Learning for End-to-End Mobile Manipulation](https://mlsanity.com/paper/2202.07600/) (2022)

## Abstract (arXiv)

This work proposes a new method to accurately complete sparse LiDAR maps guided by RGB images. For autonomous vehicles and robotics the use of LiDAR is indispensable in order to achieve precise depth predictions. A multitude of applications depend on the awareness of their surroundings, and use depth cues to reason and react accordingly. On the one hand, monocular depth prediction methods fail to generate absolute and precise depth maps. On the other hand, stereoscopic approaches are still significantly outperformed by LiDAR based approaches. The goal of the depth completion task is to generate dense depth predictions from sparse and irregular point clouds which are mapped to a 2D plane. We propose a new framework which extracts both global and local information in order to produce proper depth maps. We argue that simple depth completion does not require a deep network. However, we additionally propose a fusion method with RGB guidance from a monocular camera in order to leverage object information and to correct mistakes in the sparse input. This improves the accuracy significantly. Moreover, confidence masks are exploited in order to take into account the uncertainty in the depth predictions from each modality. This fusion method outperforms the state-of-the-art and ranks first on the KITTI depth completion benchmark. Our code with visualizations is available.