mlsanityData to 14 Oct 2026

End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection

arXiv 2004.03080 ↗First posted 7 Apr 2020cs.CV eess.IVcited by 80

Not on the map. Why?

Abstract (arXiv)

Reliable and accurate 3D object detection is a necessity for safe autonomous driving. Although LiDAR sensors can provide accurate 3D point cloud estimates of the environment, they are also prohibitively expensive for many settings. Recently, the introduction of pseudo-LiDAR (PL) has led to a drastic reduction in the accuracy gap between methods based on LiDAR sensors and those based on cheap stereo cameras. PL combines state-of-the-art deep neural networks for 3D depth estimation with those for 3D object detection by converting 2D depth map outputs to 3D point cloud inputs. However, so far these two networks have to be trained separately. In this paper, we introduce a new framework based on differentiable Change of Representation (CoR) modules that allow the entire PL pipeline to be trained end-to-end. The resulting framework is compatible with most state-of-the-art networks for both tasks and in combination with PointRCNN improves over PL consistently across all benchmarks -- yielding the highest entry on the KITTI image-based 3D object detection leaderboard at the time of submission. Our code will be made available at https://github.com/mileyan/pseudo-LiDAR_e2e.

Key facts. Cited by 80 papers in the map (data through 14 Oct 2026). First posted on arXiv 7 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 25; 2022: 22; 2023: 24; 2024: 5; 2025: 2; 2026: 2.

80 papers.

Sort:

By area:

  1. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  2. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  3. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 327
  4. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 249
  5. 2021PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detectioncited by 232
  6. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  7. 2021Multimodal Virtual Point 3D Detectioncited by 166
  8. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 162
  9. 2022Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Cloudscited by 131
  10. 2022Vision-Centric BEV Perception: A Surveycited by 126
  11. 20213D Object Detection for Autonomous Driving: A Surveycited by 106
  12. 2021AutoShape: Real-Time Shape-Aware Monocular 3D Object Detectioncited by 87
38 more
  1. 2021LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based 3D Detectorcited by 70
  2. 20223D Object Detection from Images for Autonomous Driving: A Surveycited by 66
  3. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  4. 2022Ithaca365: Dataset and Driving Perception under Repeated and Challenging Weather Conditionscited by 48
  5. 2022Diversity Matters: Fully Exploiting Depth Clues for Reliable Monocular 3D Object Detectioncited by 46
  6. 2022NeRF-RPN: A general framework for object detection in NeRFscited by 46
  7. 2023CAPE: Camera View Position Embedding for Multi-View 3D Object Detectioncited by 40
  8. 2021YOLOStereo3D: A Step Back to 2D for Efficient Stereo 3D Detectioncited by 34
  9. 2021Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overviewcited by 31
  10. 2022LET-3D-AP: Longitudinal Error Tolerant 3D Average Precision for Camera-Only 3D Detectioncited by 27
  11. 2021IPS300+: a Challenging Multimodal Dataset for Intersection Perception Systemcited by 26
  12. 2022Homography Loss for Monocular 3D Object Detectioncited by 24
  13. 2022OPA-3D: Occlusion-Aware Pixel-Wise Aggregation for Monocular 3D Object Detectioncited by 24
  14. 2021Exploiting Playbacks in Unsupervised Domain Adaptation for 3D Object Detectioncited by 21
  15. 2021PLUMENet: Efficient 3D Object Detection from Stereo Imagescited by 19
  16. 2023Adaptive Multi-Modal Cross-Entropy Loss for Stereo Matchingcited by 19
  17. 2023Fast-BEV: Towards Real-time On-vehicle Bird's-Eye View Perceptioncited by 18
  18. 2023Viewpoint Equivariance for Multi-View 3D Object Detectioncited by 18
  19. 2021MonoCInIS: Camera Independent Monocular 3D Object Detection using Instance Segmentationcited by 16
  20. 2021IAFA: Instance-aware Feature Aggregation for 3D Object Detection from a Single Imagecited by 15
  21. 2021Advancing Self-supervised Monocular Depth Learning with Sparse LiDARcited by 13
  22. 2023NeurOCS: Neural NOCS Supervision for Monocular 3D Object Localizationcited by 13
  23. 2021SIDE: Center-based Stereo 3D Detector with Structure-aware Instance Depth Estimationcited by 12
  24. 2023Unleash the Potential of Image Branch for Cross-modal 3D Object Detectioncited by 11
  25. 2021Stereo CenterNet based 3D Object Detection for Autonomous Drivingcited by 10
  26. 2022DSGN++: Exploiting Visual-Spatial Relation for Stereo-based 3D Detectorscited by 10
  27. 2023Parametric Depth Based Feature Representation Learning for Object Detection and Segmentation in Bird's Eye Viewcited by 10
  28. 2022ORA3D: Overlap Region Aware Multi-view 3D Object Detectioncited by 8
  29. 2023Lift-Attend-Splat: Bird's-eye-view camera-lidar fusion using transformerscited by 8
  30. 20223D Scene Flow Estimation on Pseudo-LiDAR: Bridging the Gap on Estimating Point Motioncited by 7
  31. 2023StereoDistill: Pick the Cream from LiDAR for Distilling Stereo-based 3D Object Detectioncited by 7
  32. 2022Dynamics-aware Adversarial Attack of Adaptive Neural Networkscited by 5
  33. 2022Depth Is All You Need for Monocular 3D Detectioncited by 4
  34. 2023Predictive World Models from Real-World Partial Observationscited by 4
  35. 2024Better Monocular 3D Detectors with LiDAR from the Pastcited by 4
  36. 2021Aug3D-RPN: Improving Monocular 3D Object Detection by Synthetic Images with Virtual Depthcited by 3
  37. 2021SM3D: Simultaneous Monocular Mapping and 3D Detectioncited by 3
  38. 2022Center Feature Fusion: Selective Multi-Sensor Fusion of Center-based Objectscited by 3

50 most cited of 80 All 80 (JSON)