# Robust and accurate depth estimation by fusing LiDAR and Stereo

arXiv: https://arxiv.org/abs/2207.06139 · HTML: https://mlsanity.com/paper/2207.06139/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 41 references that are in the map.
- First posted on arXiv 13 Jul 2022.

## Builds on (most cited first, 16 of 16)

- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches](https://mlsanity.com/paper/1510.05970/) (2015)
- [In Defense of Classical Image Processing: Fast Depth Completion on the CPU](https://mlsanity.com/paper/1802.00036/) (2018)
- [Intel RealSense Stereoscopic Depth Cameras](https://mlsanity.com/paper/1705.05548/) (2017)
- [Adaptive Context-Aware Multi-Modal Network for Depth Completion](https://mlsanity.com/paper/2008.10833/) (2020)
- [Multi-Frame Self-Supervised Depth with Transformers](https://mlsanity.com/paper/2204.07616/) (2022)
- [Unsupervised Depth Completion with Calibrated Backprojection Layers](https://mlsanity.com/paper/2108.10531/) (2021)
- [3D LiDAR and Stereo Fusion using Stereo Matching Network with Conditional Cost Volume Normalization](https://mlsanity.com/paper/1904.02917/) (2019)
- [Volumetric Propagation Network: Stereo-LiDAR Fusion for Long-Range Depth Estimation](https://mlsanity.com/paper/2103.12964/) (2021)
- [LiStereo: Generate Dense Depth Maps from LIDAR and Stereo Imagery](https://mlsanity.com/paper/1905.02744/) (2019)
- [High-resolution LIDAR-based Depth Mapping using Bilateral Filter](https://mlsanity.com/paper/1606.05614/) (2016)
- [LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR](https://mlsanity.com/paper/2109.03569/) (2021)
- [SGTBN: Generating Dense Depth Maps from Single-Line LiDAR](https://mlsanity.com/paper/2106.12994/) (2021)
- [Real Time Dense Depth Estimation by Fusing Stereo with Sparse Depth Measurements](https://mlsanity.com/paper/1809.07677/) (2018)

## Abstract (arXiv)

Depth estimation is one of the key technologies in some fields such as autonomous driving and robot navigation. However, the traditional method of using a single sensor is inevitably limited by the performance of the sensor. Therefore, a precision and robust method for fusing the LiDAR and stereo cameras is proposed. This method fully combines the advantages of the LiDAR and stereo camera, which can retain the advantages of the high precision of the LiDAR and the high resolution of images respectively. Compared with the traditional stereo matching method, the texture of the object and lighting conditions have less influence on the algorithm. Firstly, the depth of the LiDAR data is converted to the disparity of the stereo camera. Because the density of the LiDAR data is relatively sparse on the y-axis, the converted disparity map is up-sampled using the interpolation method. Secondly, in order to make full use of the precise disparity map, the disparity map and stereo matching are fused to propagate the accurate disparity. Finally, the disparity map is converted to the depth map. Moreover, the converted disparity map can also increase the speed of the algorithm. We evaluate the proposed pipeline on the KITTI benchmark. The experiment demonstrates that our algorithm has higher accuracy than several classic methods.