mlsanityData to 14 Oct 2026

Better Monocular 3D Detectors with LiDAR from the Past

arXiv 2404.05139 ↗First posted 8 Apr 2024cs.CV cs.ROcited by 4

Not on the map. Why?

Abstract (arXiv)

Accurate 3D object detection is crucial to autonomous driving. Though LiDAR-based detectors have achieved impressive performance, the high cost of LiDAR sensors precludes their widespread adoption in affordable vehicles. Camera-based detectors are cheaper alternatives but often suffer inferior performance compared to their LiDAR-based counterparts due to inherent depth ambiguities in images. In this work, we seek to improve monocular 3D detectors by leveraging unlabeled historical LiDAR data. Specifically, at inference time, we assume that the camera-based detectors have access to multiple unlabeled LiDAR scans from past traversals at locations of interest (potentially from other high-end vehicles equipped with LiDAR sensors). Under this setup, we proposed a novel, simple, and end-to-end trainable framework, termed AsyncDepth, to effectively extract relevant features from asynchronous LiDAR traversals of the same location for monocular 3D detectors. We show consistent and significant performance gain (up to 9 AP) across multiple state-of-the-art models and datasets with a negligible additional latency of 9.66 ms and a small storage cost.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 46 of its 59 references that are in the map. First posted on arXiv 8 Apr 2024.

Builds on

46 of 59 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  7. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  8. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  9. 2017Deformable Convolutional Networkscited by 1,488
  10. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  11. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  12. 2020Center-based 3D Object Detection and Trackingcited by 1,144
34 more
  1. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  2. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
  3. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  4. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  5. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  6. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  7. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  8. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 466
  9. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
  10. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  11. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  12. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  13. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 425
  14. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 352
  15. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 327
  16. 2019M3D-RPN: Monocular 3D Region Proposal Network for Object Detectioncited by 252
  17. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 249
  18. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 191
  19. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  20. 2019Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Drivingcited by 188
  21. 2019Stereo R-CNN based 3D Object Detection for Autonomous Drivingcited by 163
  22. 2022M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representationcited by 156
  23. 2022Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Datacited by 126
  24. 2018Deep Depth Completion of a Single RGB-D Imagecited by 110
  25. 2019Depth Completion from Sparse LiDAR Data with Depth-Normal Constraintscited by 108
  26. 2020End-to-End Pseudo-LiDAR for Image-Based 3D Object Detectioncited by 80
  27. 2020Self-training for Few-shot Transfer Across Extreme Task Differencescited by 76
  28. 2022Ithaca365: Dataset and Driving Perception under Repeated and Challenging Weather Conditionscited by 48
  29. 2022Learning to Detect Mobile Objects from LiDAR Scans Without Labelscited by 44
  30. 2022Graph-DETR3D: Rethinking Overlapping Regions for Multi-View 3D Object Detectioncited by 35
  31. 2017Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environmentscited by 22
  32. 2022Hindsight is 20/20: Leveraging Past Traversals to Aid 3D Perceptioncited by 20
  33. 2021PLUMENet: Efficient 3D Object Detection from Stereo Imagescited by 19
  34. 2023Unsupervised Adaptation from Repeated Traversals for Autonomous Drivingcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 2.

4 papers.

  1. 2024Memorize What Matters: Emergent Scene Decomposition from Multitraversecited by 8
  2. 2025Spatial Retrieval Augmented Autonomous Drivingcited by 5
  3. 2026Leveraging Previous-Traversal Point Cloud Map Priors for Camera-Based 3D Object Detection and Trackingcited by 2
  4. 2026Leveraging Vision-Based Point Cloud Map Priors for Camera-Based 3D Object Detection and Online Vectorized HD Mappingcited by 0