# Sparse LiDAR and Stereo Fusion (SLS-Fusion) for Depth Estimationand 3D Object Detection

arXiv: https://arxiv.org/abs/2103.03977 · HTML: https://mlsanity.com/paper/2103.03977/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 27 references that are in the map.
- First posted on arXiv 5 Mar 2021.

## Builds on (most cited first, 23 of 23)

- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [PIXOR: Real-time 3D Object Detection from Point Clouds](https://mlsanity.com/paper/1902.06326/) (2019)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/1906.06310/) (2019)
- [Stereo R-CNN based 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1902.09738/) (2019)
- [DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image](https://mlsanity.com/paper/1812.00488/) (2018)
- [Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection](https://mlsanity.com/paper/1903.01864/) (2019)
- [Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud](https://mlsanity.com/paper/1903.09847/) (2019)
- [DSGN: Deep Stereo Geometry Network for 3D Object Detection](https://mlsanity.com/paper/2001.03398/) (2020)
- [Deep MANTA: A Coarse-to-fine Many-Task Network for joint 2D and 3D vehicle analysis from monocular image](https://mlsanity.com/paper/1703.07570/) (2017)
- [Triangulation Learning Network: from Monocular to Stereo 3D Object Detection](https://mlsanity.com/paper/1906.01193/) (2019)
- [Noise-Aware Unsupervised Deep Lidar-Stereo Fusion](https://mlsanity.com/paper/1904.03868/) (2019)
- [Object-Centric Stereo Matching for 3D Object Detection](https://mlsanity.com/paper/1909.07566/) (2019)
- [Task-Aware Monocular Depth Estimation for 3D Object Detection](https://mlsanity.com/paper/1909.07701/) (2019)
- [Confidence Guided Stereo 3D Object Detection with Split Depth Estimation](https://mlsanity.com/paper/2003.05505/) (2020)
- [3D LiDAR and Stereo Fusion using Stereo Matching Network with Conditional Cost Volume Normalization](https://mlsanity.com/paper/1904.02917/) (2019)

## Built on by (most cited first, 3 of 3)

- [ORA3D: Overlap Region Aware Multi-view 3D Object Detection](https://mlsanity.com/paper/2207.00865/) (2022)
- [Stereo-LiDAR Depth Estimation with Deformable Propagation and Learned Disparity-Depth Conversion](https://mlsanity.com/paper/2404.07545/) (2024)
- [Preserving Guidance in Cost-Volume Retrieval under Extreme LiDAR Sparsity in Iterative Stereo](https://mlsanity.com/paper/2507.19738/) (2025)

## Abstract (arXiv)

The ability to accurately detect and localize objects is recognized as being the most important for the perception of self-driving cars. From 2D to 3D object detection, the most difficult is to determine the distance from the ego-vehicle to objects. Expensive technology like LiDAR can provide a precise and accurate depth information, so most studies have tended to focus on this sensor showing a performance gap between LiDAR-based methods and camera-based methods. Although many authors have investigated how to fuse LiDAR with RGB cameras, as far as we know there are no studies to fuse LiDAR and stereo in a deep neural network for the 3D object detection task. This paper presents SLS-Fusion, a new approach to fuse data from 4-beam LiDAR and a stereo camera via a neural network for depth estimation to achieve better dense depth maps and thereby improves 3D object detection performance. Since 4-beam LiDAR is cheaper than the well-known 64-beam LiDAR, this approach is also classified as a low-cost sensors-based method. Through evaluation on the KITTI benchmark, it is shown that the proposed method significantly improves depth estimation performance compared to a baseline method. Also, when applying it to 3D object detection, a new state of the art on low-cost sensor based method is achieved.