# PLUMENet: Efficient 3D Object Detection from Stereo Images

arXiv: https://arxiv.org/abs/2101.06594 · HTML: https://mlsanity.com/paper/2101.06594/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 47 references that are in the map.
- First posted on arXiv 17 Jan 2021.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [PIXOR: Real-time 3D Object Detection from Point Clouds](https://mlsanity.com/paper/1902.06326/) (2019)
- [Point-Voxel CNN for Efficient 3D Deep Learning](https://mlsanity.com/paper/1907.03739/) (2019)
- [3D Bounding Box Estimation Using Deep Learning and Geometry](https://mlsanity.com/paper/1612.00496/) (2016)
- [Deep Continuous Fusion for Multi-Sensor 3D Object Detection](https://mlsanity.com/paper/2012.10992/) (2020)
- [Multi-Task Multi-Sensor Fusion for 3D Object Detection](https://mlsanity.com/paper/2012.12397/) (2020)
- [Orthographic Feature Transform for Monocular 3D Object Detection](https://mlsanity.com/paper/1811.08188/) (2018)
- [Learning a Multi-View Stereo Machine](https://mlsanity.com/paper/1708.05375/) (2017)
- [Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/1906.06310/) (2019)
- [Stereo R-CNN based 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1902.09738/) (2019)
- [Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches](https://mlsanity.com/paper/1510.05970/) (2015)
- [GS3D: An Efficient 3D Object Detection Framework for Autonomous Driving](https://mlsanity.com/paper/1903.10955/) (2019)
- [DSGN: Deep Stereo Geometry Network for 3D Object Detection](https://mlsanity.com/paper/2001.03398/) (2020)
- [Deep MANTA: A Coarse-to-fine Many-Task Network for joint 2D and 3D vehicle analysis from monocular image](https://mlsanity.com/paper/1703.07570/) (2017)
- [End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection](https://mlsanity.com/paper/2004.03080/) (2020)
- [HDNET: Exploiting HD Maps for 3D Object Detection](https://mlsanity.com/paper/2012.11704/) (2020)
- [3D Object Proposals using Stereo Imagery for Accurate Object Class Detection](https://mlsanity.com/paper/1608.07711/) (2016)

## Built on by (most cited first, 19 of 19)

- [3D Object Detection for Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2206.09474/) (2022)
- [LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based 3D Detector](https://mlsanity.com/paper/2108.08258/) (2021)
- [VPFNet: Improving 3D Object Detection with Virtual Point based LiDAR and Stereo Data Fusion](https://mlsanity.com/paper/2111.14382/) (2021)
- [Pseudo-Stereo for Monocular 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/2203.02112/) (2022)
- [ORA3D: Overlap Region Aware Multi-view 3D Object Detection](https://mlsanity.com/paper/2207.00865/) (2022)
- [Weak-to-Strong 3D Object Detection with X-Ray Distillation](https://mlsanity.com/paper/2404.00679/) (2024)
- [Better Monocular 3D Detectors with LiDAR from the Past](https://mlsanity.com/paper/2404.05139/) (2024)
- [Stereo Neural Vernier Caliper](https://mlsanity.com/paper/2203.11018/) (2022)
- [Consistency of Implicit and Explicit Features Matters for Monocular 3D Object Detection](https://mlsanity.com/paper/2207.07933/) (2022)
- [FFPA-Net: Efficient Feature Fusion with Projection Awareness for 3D Object Detection](https://mlsanity.com/paper/2209.07419/) (2022)
- [Transformer-based stereo-aware 3D object detection from binocular images](https://mlsanity.com/paper/2304.11906/) (2023)
- [Training an Open-Vocabulary Monocular 3D Object Detection Model without 3D Data](https://mlsanity.com/paper/2411.15657/) (2024)
- [Scalable Primitives for Generalized Sensor Fusion in Autonomous Vehicles](https://mlsanity.com/paper/2112.00219/) (2021)
- [ALPI: Auto-Labeller with Proxy Injection for 3D Object Detection using 2D Labels Only](https://mlsanity.com/paper/2407.17197/) (2024)
- [Real-time Stereo-based 3D Object Detection for Streaming Perception](https://mlsanity.com/paper/2410.12394/) (2024)
- [CA-W3D: Leveraging Context-Aware Knowledge for Weakly Supervised Monocular 3D Detection](https://mlsanity.com/paper/2503.04154/) (2025)
- [An Efficient Convex Hull-based Vehicle Pose Estimation Method for 3D LiDAR](https://mlsanity.com/paper/2302.01034/) (2023)
- [Camera Movement Estimation and Path Correction using the Combination of Modified A-SIFT and Stereo System for 3D Modelling](https://mlsanity.com/paper/2503.17668/) (2025)
- [Monocular Depth Estimation from a Single Image: Progress and Opportunities](https://mlsanity.com/paper/2609.01172/) (2026)

## Abstract (arXiv)

3D object detection is a key component of many robotic applications such as self-driving vehicles. While many approaches rely on expensive 3D sensors such as LiDAR to produce accurate 3D estimates, methods that exploit stereo cameras have recently shown promising results at a lower cost. Existing approaches tackle this problem in two steps: first depth estimation from stereo images is performed to produce a pseudo LiDAR point cloud, which is then used as input to a 3D object detector. However, this approach is suboptimal due to the representation mismatch, as the two tasks are optimized in two different metric spaces. In this paper we propose a model that unifies these two tasks and performs them in the same metric space. Specifically, we directly construct a pseudo LiDAR feature volume (PLUME) in 3D space, which is then used to solve both depth estimation and object detection tasks. Our approach achieves state-of-the-art performance with much faster inference times when compared to existing methods on the challenging KITTI benchmark.