# Accurate and Real-time Pseudo Lidar Detection: Is Stereo Neural Network Really Necessary?

arXiv: https://arxiv.org/abs/2206.13858 · HTML: https://mlsanity.com/paper/2206.13858/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 35 references that are in the map.
- First posted on arXiv 28 Jun 2022.

## Builds on (most cited first, 28 of 28)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [Voxel Transformer for 3D Object Detection](https://mlsanity.com/paper/2109.02497/) (2021)
- [AANet: Adaptive Aggregation Network for Efficient Stereo Matching](https://mlsanity.com/paper/2004.09548/) (2020)
- [Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/1906.06310/) (2019)
- [Hierarchical Neural Architecture Search for Deep Stereo Matching](https://mlsanity.com/paper/2010.13501/) (2020)
- [Stereo R-CNN based 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1902.09738/) (2019)
- [Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches](https://mlsanity.com/paper/1510.05970/) (2015)
- [SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud](https://mlsanity.com/paper/2104.09804/) (2021)
- [Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud](https://mlsanity.com/paper/1903.09847/) (2019)
- [Rethinking Pseudo-LiDAR Representation](https://mlsanity.com/paper/2008.04582/) (2020)
- [SPG: Unsupervised Domain Adaptation for 3D Object Detection via Semantic Point Generation](https://mlsanity.com/paper/2108.06709/) (2021)
- [End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection](https://mlsanity.com/paper/2004.03080/) (2020)
- [Disp R-CNN: Stereo 3D Object Detection via Shape Prior Guided Instance Disparity Estimation](https://mlsanity.com/paper/2004.03572/) (2020)
- [Triangulation Learning Network: from Monocular to Stereo 3D Object Detection](https://mlsanity.com/paper/1906.01193/) (2019)
- [Object-Centric Stereo Matching for 3D Object Detection](https://mlsanity.com/paper/1909.07566/) (2019)
- [ZoomNet: Part-Aware Adaptive Zooming Neural Network for 3D Object Detection](https://mlsanity.com/paper/2003.00529/) (2020)
- [A Decomposition Model for Stereo Matching](https://mlsanity.com/paper/2104.07516/) (2021)
- [RTS3D: Real-time Stereo 3D Detection from 4D Feature-Consistency Embedding Space for Autonomous Driving](https://mlsanity.com/paper/2012.15072/) (2020)
- [Embedded real-time stereo estimation via Semi-Global Matching on the GPU](https://mlsanity.com/paper/1610.04121/) (2016)

## Built on by (most cited first, 1 of 1)

- [Progressive Bird's Eye View Perception for Safety-Critical Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2508.07560/) (2025)

## Abstract (arXiv)

The proposal of Pseudo-Lidar representation has significantly narrowed the gap between visual-based and active Lidar-based 3D object detection. However, current researches exclusively focus on pushing the accuracy improvement of Pseudo-Lidar by taking the advantage of complex and time-consuming neural networks. Seldom explore the profound characteristics of Pseudo-Lidar representation to obtain the promoting opportunities. In this paper, we dive deep into the pseudo Lidar representation and argue that the performance of 3D object detection is not fully dependent on the high precision stereo depth estimation. We demonstrate that even for the unreliable depth estimation, with proper data processing and refining, it can achieve comparable 3D object detection accuracy. With this finding, we further show the possibility that utilizing fast but inaccurate stereo matching algorithms in the Pseudo-Lidar system to achieve low latency responsiveness. In the experiments, we develop a system with a less powerful stereo matching predictor and adopt the proposed refinement schemes to improve the accuracy. The evaluation on the KITTI benchmark shows that the presented system achieves competitive accuracy to the state-of-the-art approaches with only 23 ms computing, showing it is a suitable candidate for deploying to real car-hold applications.