mlsanityData to 7 Oct 2026

Is Pseudo-Lidar needed for Monocular 3D Object detection?

arXiv 2108.06417 ↗First posted 13 Aug 2021cs.CVcited by 248

Not on the map. Why?

Abstract (arXiv)

Recent progress in 3D object detection from single images leverages monocular depth estimation as a way to produce 3D pointclouds, turning cameras into pseudo-lidar sensors. These two-stage detectors improve with the accuracy of the intermediate depth estimation network, which can itself be improved without manual labels via large-scale self-supervised learning. However, they tend to suffer from overfitting more than end-to-end methods, are more complex, and the gap with similar lidar-based detectors remains significant. In this work, we propose an end-to-end, single stage, monocular 3D object detector, DD3D, that can benefit from depth pre-training like pseudo-lidar methods, but without their limitations. Our architecture is designed for effective information transfer between depth estimation and 3D detection, allowing us to scale with the amount of unlabeled pre-training data. Our method achieves state-of-the-art results on two challenging benchmarks, with 16.34% and 9.28% AP for Cars and Pedestrians (respectively) on the KITTI-3D benchmark, and 41.5% mAP on NuScenes.

Key facts. Cited by 248 papers in the map (data through 7 Oct 2026). Builds on 61 of its 76 references that are in the map. First posted on arXiv 13 Aug 2021.

Builds on

61 of 76 references.

References · 61

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  5. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  6. 2017Squeeze-and-Excitation Networkscited by 4,763
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  9. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  10. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  11. 2019Objects as Pointscited by 1,172
  12. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
38 more
  1. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  2. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  3. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  4. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  5. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  6. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  7. 2017Deep Layer Aggregationcited by 436
  8. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 402
  9. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  10. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  11. 20163D Bounding Box Estimation Using Deep Learning and Geometrycited by 277
  12. 2019M3D-RPN: Monocular 3D Region Proposal Network for Object Detectioncited by 251
  13. 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
  14. 2019Disentangling Monocular 3D Object Detectioncited by 215
  15. 2016UnitBox: An Advanced Object Detection Networkcited by 205
  16. 2020SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimationcited by 202
  17. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 201
  18. 2016DeMoN: Depth and Motion Network for Learning Monocular Stereocited by 200
  19. 2019CenterMask : Real-Time Anchor-Free Instance Segmentationcited by 200
  20. 2019Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Drivingcited by 188
  21. 2019Learning Depth-Guided Convolutions for Monocular 3D Object Detectioncited by 169
  22. 2020MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationshipscited by 153
  23. 2020Train in Germany, Test in The USA: Making 3D Object Detectors Generalizecited by 150
  24. 2020Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidancecited by 148
  25. 2019Accurate Monocular Object Detection via Color-Embedded 3D Reconstruction for Autonomous Drivingcited by 137
  26. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  27. 2018ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shapecited by 123
  28. 2020Kinematic 3D Object Detection in Monocular Videocited by 116
  29. 2019Monocular 3D Object Detection with Pseudo-LiDAR Point Cloudcited by 113
  30. 2020Rethinking Pseudo-LiDAR Representationcited by 109
  31. 2019GS3D: An Efficient 3D Object Detection Framework for Autonomous Drivingcited by 106
  32. 2019Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstructioncited by 106
  33. 2017Deep MANTA: A Coarse-to-fine Many-Task Network for joint 2D and 3D vehicle analysis from monocular imagecited by 95
  34. 2017SfM-Net: Learning of Structure and Motion from Videocited by 89
  35. 2018MonoGRNet: A Geometric Reasoning Network for Monocular 3D Object Localizationcited by 84
  36. 2020End-to-End Pseudo-LiDAR for Image-Based 3D Object Detectioncited by 80
  37. 2019Deep Fitting Degree Scoring Network for Monocular 3D Object Detectioncited by 67
  38. 2018SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimationcited by 64

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 70; 2023: 72; 2024: 51; 2025: 37; 2026: 12.

248 papers.

Sort:

By area:

  1. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  2. 2022Planning-oriented Autonomous Drivingcited by 1,068
  3. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  4. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  5. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  6. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  7. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  8. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  9. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  10. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
  11. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  12. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
38 more
  1. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  2. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  3. 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 244
  4. 2022Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representationcited by 243
  5. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  6. 2022FUTR3D: A Unified Sensor Fusion Framework for 3D Detectioncited by 194
  7. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 192
  8. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  9. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
  10. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 161
  11. 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 159
  12. 2022M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representationcited by 155
  13. 2023SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videoscited by 149
  14. 2022PolarFormer: Multi-camera 3D Object Detection with Polar Transformercited by 147
  15. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  16. 2022Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?cited by 128
  17. 2022Vision-Centric BEV Perception: A Surveycited by 126
  18. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  19. 2023FB-BEV: BEV Representation from Forward-Backward View Transformationscited by 98
  20. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 97
  21. 2023Sparse4D v2: Recurrent Temporal Fusion with Sparse Modelcited by 92
  22. 2022MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queriescited by 90
  23. 2022MonoDETR: Depth-guided Transformer for Monocular 3D Object Detectioncited by 85
  24. 2023CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perceptioncited by 81
  25. 2023EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimationcited by 79
  26. 2023UniPAD: A Universal Pre-training Paradigm for Autonomous Drivingcited by 73
  27. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  28. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 69
  29. 20223D Object Detection from Images for Autonomous Driving: A Surveycited by 66
  30. 2022CRAFT: Camera-Radar 3D Object Detection with Spatio-Contextual Fusion Transformercited by 66
  31. 2022BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detectioncited by 66
  32. 2025Generalized Trajectory Scoring for End-to-end Multimodal Planningcited by 63
  33. 2025UniK3D: Universal Camera Monocular 3D Estimationcited by 60
  34. 2023PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigmcited by 56
  35. 2023Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Trackingcited by 54
  36. 2022Polar Parametrization for Vision-based Surround-View 3D Detectioncited by 53
  37. 2021EPNet++: Cascade Bi-directional Fusion for Multi-Modal 3D Object Detectioncited by 49
  38. 2022Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detectioncited by 49

50 most cited of 248 All 248 (JSON)