# LET-3D-AP: Longitudinal Error Tolerant 3D Average Precision for Camera-Only 3D Detection

arXiv: https://arxiv.org/abs/2206.07705 · HTML: https://mlsanity.com/paper/2206.07705/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 68 references that are in the map.
- First posted on arXiv 15 Jun 2022.

## Builds on (most cited first, 30 of 46)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection](https://mlsanity.com/paper/2104.10956/) (2021)
- [PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images](https://mlsanity.com/paper/2206.01256/) (2022)
- [BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection](https://mlsanity.com/paper/2203.17054/) (2022)
- [Categorical Depth Distribution Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.01100/) (2021)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [3D Bounding Box Estimation Using Deep Learning and Geometry](https://mlsanity.com/paper/1612.00496/) (2016)
- [M3D-RPN: Monocular 3D Region Proposal Network for Object Detection](https://mlsanity.com/paper/1907.06038/) (2019)
- [Disentangling Monocular 3D Object Detection](https://mlsanity.com/paper/1905.12365/) (2019)
- [Orthographic Feature Transform for Monocular 3D Object Detection](https://mlsanity.com/paper/1811.08188/) (2018)
- [BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Driving](https://mlsanity.com/paper/2205.09743/) (2022)
- [Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/1906.06310/) (2019)
- [Learning Depth-Guided Convolutions for Monocular 3D Object Detection](https://mlsanity.com/paper/1912.04799/) (2019)
- [MVX-Net: Multimodal VoxelNet for 3D Object Detection](https://mlsanity.com/paper/1904.01649/) (2019)
- [ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection](https://mlsanity.com/paper/2106.01178/) (2021)
- [M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representation](https://mlsanity.com/paper/2204.05088/) (2022)

## Built on by (most cited first, 27 of 27)

- [EMMA: End-to-End Multimodal Model for Autonomous Driving](https://mlsanity.com/paper/2410.23262/) (2024)
- [World Simulation with Video Foundation Models for Physical AI](https://mlsanity.com/paper/2511.00062/) (2025)
- [Vision-Centric BEV Perception: A Survey](https://mlsanity.com/paper/2208.02797/) (2022)
- [Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe](https://mlsanity.com/paper/2209.05324/) (2022)
- [Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Models](https://mlsanity.com/paper/2506.09042/) (2025)
- [CRAFT: Camera-Radar 3D Object Detection with Spatio-Contextual Fusion Transformer](https://mlsanity.com/paper/2209.06535/) (2022)
- [3D and 4D World Modeling: A Survey](https://mlsanity.com/paper/2509.07996/) (2025)
- [BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2211.09386/) (2022)
- [FreeVS: Generative View Synthesis on Free Driving Trajectory](https://mlsanity.com/paper/2410.18079/) (2024)
- [MV-FCOS3D++: Multi-View Camera-Only 4D Object Detection with Pretrained Monocular Backbones](https://mlsanity.com/paper/2207.12716/) (2022)
- [X3KD: Knowledge Distillation Across Modalities, Tasks and Stages for Multi-Camera 3D Object Detection](https://mlsanity.com/paper/2303.02203/) (2023)
- [Echoes Beyond Points: Unleashing the Power of Raw Radar Data in Multi-modality Fusion](https://mlsanity.com/paper/2307.16532/) (2023)
- [GeoMIM: Towards Better 3D Knowledge Transfer via Masked Image Modeling for Multi-view 3D Understanding](https://mlsanity.com/paper/2303.11325/) (2023)
- [3D Data Augmentation for Driving Scenes on Camera](https://mlsanity.com/paper/2303.10340/) (2023)
- [Far3Det: Towards Far-Field 3D Detection](https://mlsanity.com/paper/2211.13858/) (2022)
- [3D Video Object Detection with Learnable Object-Centric Global Optimization](https://mlsanity.com/paper/2303.15416/) (2023)
- [Realistic and Controllable 3D Gaussian-Guided Object Editing for Driving Video Generation](https://mlsanity.com/paper/2508.20471/) (2025)
- [Learning High-resolution Vector Representation from Multi-Camera Images for 3D Object Detection](https://mlsanity.com/paper/2407.15354/) (2024)
- [RC-BEVFusion: A Plug-In Module for Radar-Camera Bird's Eye View Feature Fusion](https://mlsanity.com/paper/2305.15883/) (2023)
- [Weakly Supervised 3D Object Detection with Multi-Stage Generalization](https://mlsanity.com/paper/2306.05418/) (2023)
- [Instance-Level Safety-Aware Fidelity of Synthetic Data and Its Calibration](https://mlsanity.com/paper/2402.07031/) (2024)
- [EC-IoU: Orienting Safety for Object Detectors via Ego-Centric Intersection-over-Union](https://mlsanity.com/paper/2403.15474/) (2024)
- [nuScenes Revisited: Progress and Challenges in Autonomous Driving](https://mlsanity.com/paper/2512.02448/) (2025)
- [DenseBEV: Transforming BEV Grid Cells into 3D Objects](https://mlsanity.com/paper/2512.16818/) (2025)
- [TopView: Vectorising road users in a bird's eye view from uncalibrated street-level imagery with deep learning](https://mlsanity.com/paper/2412.16229/) (2024)
- [Rethink 3D Object Detection from Physical World](https://mlsanity.com/paper/2507.00190/) (2025)
- [StixelNExT++: Lightweight Monocular Scene Segmentation and Representation for Collective Perception](https://mlsanity.com/paper/2507.06687/) (2025)

## Abstract (arXiv)

The 3D Average Precision (3D AP) relies on the intersection over union between predictions and ground truth objects. However, camera-only detectors have limited depth accuracy, which may cause otherwise reasonable predictions that suffer from such longitudinal localization errors to be treated as false positives. We therefore propose variants of the 3D AP metric to be more permissive with respect to depth estimation errors. Specifically, our novel longitudinal error tolerant metrics, LET-3D-AP and LET-3D-APL, allow longitudinal localization errors of the prediction boxes up to a given tolerance. To evaluate the proposed metrics, we also construct a new test set for the Waymo Open Dataset, tailored to camera-only 3D detection methods. Surprisingly, we find that state-of-the-art camera-based detectors can outperform popular LiDAR-based detectors with our new metrics past at 10% depth error tolerance, suggesting that existing camera-based detectors already have the potential to surpass LiDAR-based detectors in downstream applications. We believe the proposed metrics and the new benchmark dataset will facilitate advances in the field of camera-only 3D detection by providing more informative signals that can better indicate the system-level performance.