# Towards Safe, Real-Time Systems: Stereo vs Images and LiDAR for 3D Object Detection

arXiv: https://arxiv.org/abs/2202.12773 · HTML: https://mlsanity.com/paper/2202.12773/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 17 references that are in the map.
- First posted on arXiv 25 Feb 2022.

## Builds on (most cited first, 11 of 11)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [Stereo R-CNN based 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1902.09738/) (2019)
- [Learning Rich Features from RGB-D Images for Object Detection and Segmentation](https://mlsanity.com/paper/1407.5736/) (2014)
- [DSGN: Deep Stereo Geometry Network for 3D Object Detection](https://mlsanity.com/paper/2001.03398/) (2020)
- [Hierarchical Discrete Distribution Decomposition for Match Density Estimation](https://mlsanity.com/paper/1812.06264/) (2018)
- [3D Object Proposals using Stereo Imagery for Accurate Object Class Detection](https://mlsanity.com/paper/1608.07711/) (2016)
- [Disp R-CNN: Stereo 3D Object Detection via Shape Prior Guided Instance Disparity Estimation](https://mlsanity.com/paper/2004.03572/) (2020)
- [YOLO3D: End-to-end real-time 3D Oriented Object Bounding Box Detection from LiDAR Point Cloud](https://mlsanity.com/paper/1808.02350/) (2018)
- [Tiny SSD: A Tiny Single-shot Detection Deep Convolutional Neural Network for Real-time Embedded Object Detection](https://mlsanity.com/paper/1802.06488/) (2018)
- [LRPD: Long Range 3D Pedestrian Detection Leveraging Specific Strengths of LiDAR and RGB](https://mlsanity.com/paper/2006.09738/) (2020)

## Abstract (arXiv)

As object detectors rapidly improve, attention has expanded past image-only networks to include a range of 3D and multimodal frameworks, especially ones that incorporate LiDAR. However, due to cost, logistics, and even some safety considerations, stereo can be an appealing alternative. Towards understanding the efficacy of stereo as a replacement for monocular input or LiDAR in object detectors, we show that multimodal learning with traditional disparity algorithms can improve image-based results without increasing the number of parameters, and that learning over stereo error can impart similar 3D localization power to LiDAR in certain contexts. Furthermore, doing so also has calibration benefits with respect to image-only methods. We benchmark on the public dataset KITTI, and in doing so, reveal a few small but common algorithmic mistakes currently used in computing metrics on that set, and offer efficient, provably correct alternatives.