mlsanityData to 14 Oct 2026

Depth Is All You Need for Monocular 3D Detection

arXiv 2210.02493 ↗First posted 5 Oct 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

A key contributor to recent progress in 3D detection from single images is monocular depth estimation. Existing methods focus on how to leverage depth explicitly, by generating pseudo-pointclouds or providing attention cues for image features. More recent works leverage depth prediction as a pretraining task and fine-tune the depth representation while training it for 3D detection. However, the adaptation is insufficient and is limited in scale by manual labels. In this work, we propose to further align depth representation with the target domain in unsupervised fashions. Our methods leverage commonly available LiDAR or RGB videos during training time to fine-tune the depth representation, which leads to improved 3D detectors. Especially when using RGB videos, we show that our two-stage training by first generating pseudo-depth labels is critical because of the inconsistency in loss distribution between the two tasks. With either type of reference data, our multi-task learning approach improves over the state of the art on both KITTI and NuScenes, while matching the test-time complexity of its single task sub-network.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 50 of its 58 references that are in the map. First posted on arXiv 5 Oct 2022.

Builds on

50 of 58 references.

References · 50

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  7. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  8. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  9. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  10. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  11. 2019Objects as Pointscited by 1,175
  12. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
38 more
  1. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
  2. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  3. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  4. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  5. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
  6. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  7. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 439
  8. 2016Virtual Worlds as Proxy for Multi-Object Tracking Analysiscited by 341
  9. 2019M3D-RPN: Monocular 3D Region Proposal Network for Object Detectioncited by 252
  10. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 249
  11. 2019Disentangling Monocular 3D Object Detectioncited by 217
  12. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 211
  13. 2020SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimationcited by 204
  14. 2019CenterMask : Real-Time Anchor-Free Instance Segmentationcited by 201
  15. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 191
  16. 2019Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Drivingcited by 188
  17. 2021Objects are Different: Flexible Monocular 3D Object Detectioncited by 185
  18. 2019Learning Depth-Guided Convolutions for Monocular 3D Object Detectioncited by 169
  19. 2021Geometry Uncertainty Projection Network for Monocular 3D Object Detectioncited by 156
  20. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
  21. 2020MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationshipscited by 153
  22. 2020Train in Germany, Test in The USA: Making 3D Object Detectors Generalizecited by 151
  23. 2019Accurate Monocular Object Detection via Color-Embedded 3D Reconstruction for Autonomous Drivingcited by 137
  24. 2018ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shapecited by 123
  25. 2020Kinematic 3D Object Detection in Monocular Videocited by 116
  26. 2019Monocular 3D Object Detection with Pseudo-LiDAR Point Cloudcited by 113
  27. 2020Rethinking Pseudo-LiDAR Representationcited by 109
  28. 2022MonoDTR: Monocular 3D Object Detection with Depth-Aware Transformercited by 109
  29. 2019Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstructioncited by 106
  30. 2017Deep MANTA: A Coarse-to-fine Many-Task Network for joint 2D and 3D vehicle analysis from monocular imagecited by 95
  31. 2020End-to-End Pseudo-LiDAR for Image-Based 3D Object Detectioncited by 80
  32. 2021Monocular 3D Object Detection: An Extrinsic Parameter Free Approachcited by 69
  33. 2022Diversity Matters: Fully Exploiting Depth Clues for Reliable Monocular 3D Object Detectioncited by 46
  34. 2019Monocular 3D Object Detection via Geometric Reasoning on Keypointscited by 31
  35. 2022Pseudo-Stereo for Monocular 3D Object Detection in Autonomous Drivingcited by 30
  36. 2019Robust Semi-Supervised Monocular Depth Estimation with Reprojected Distancescited by 27
  37. 2019RefinedMPL: Refined Monocular PseudoLiDAR for 3D Object Detection in Autonomous Drivingcited by 23
  38. 2018The Earth ain't Flat: Monocular Reconstruction of Vehicles on Steep and Graded Roads from a Moving Cameracited by 11

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 1.

4 papers.

  1. 2024MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detectioncited by 10
  2. 2024FlatFusion: Delving into Details of Sparse Transformer-based Camera-LiDAR Fusion for Autonomous Drivingcited by 9
  3. 2024Multi-View Attentive Contextualization for Multi-View 3D Object Detectioncited by 7
  4. 2025MonoCT: Overcoming Monocular 3D Detection Domain Shift with Consistent Teacher Modelscited by 2