# On Offline Evaluation of 3D Object Detection for Autonomous Driving

arXiv: https://arxiv.org/abs/2308.12779 · HTML: https://mlsanity.com/paper/2308.12779/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in VLMs for autonomous driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 23 of its 34 references that are in the map.
- First posted on arXiv 24 Aug 2023.

## Placed in

- End-to-end autonomous driving / [VLMs for autonomous driving](https://mlsanity.com/driving/vlms-for-autonomous-driving/)

## Builds on (most cited first, 23 of 23)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles](https://mlsanity.com/paper/2106.11810/) (2021)
- [TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving](https://mlsanity.com/paper/2205.15997/) (2022)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [Parting with Misconceptions about Learning-based Vehicle Motion Planning](https://mlsanity.com/paper/2306.07962/) (2023)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection](https://mlsanity.com/paper/2102.00463/) (2021)
- [3D Object Detection for Autonomous Driving: A Survey](https://mlsanity.com/paper/2106.10823/) (2021)
- [PlanT: Explainable Planning Transformers via Object-Level Representations](https://mlsanity.com/paper/2210.14222/) (2022)
- [PillarNet: Real-Time and High-Performance Pillar-based 3D Object Detection](https://mlsanity.com/paper/2205.07403/) (2022)
- [On Offline Evaluation of Vision-based Driving Models](https://mlsanity.com/paper/1809.04843/) (2018)
- [Learning to Evaluate Perception Models Using Planner-Centric Metrics](https://mlsanity.com/paper/2004.08745/) (2020)
- [Can Offline Testing of Deep Neural Networks Replace Their Online Testing?](https://mlsanity.com/paper/2101.11118/) (2021)
- [Injecting Planning-Awareness into Prediction and Detection Evaluation](https://mlsanity.com/paper/2110.03270/) (2021)
- [Revisiting 3D Object Detection From an Egocentric Perspective](https://mlsanity.com/paper/2112.07787/) (2021)
- [Risk Ranked Recall: Collision Safety Metric for Object Detection Systems in Autonomous Vehicles](https://mlsanity.com/paper/2106.04146/) (2021)
- [Transcendental Idealism of Planner: Evaluating Perception from Planning Perspective for Autonomous Driving](https://mlsanity.com/paper/2306.07276/) (2023)

## Built on by (most cited first, 11 of 11)

- [Pseudo-Simulation for Autonomous Driving](https://mlsanity.com/paper/2506.04218/) (2025)
- [Label-Efficient 3D Object Detection For Road-Side Units](https://mlsanity.com/paper/2404.06256/) (2024)
- [VALO: A Versatile Anytime Framework for LiDAR-based Object Detection Deep Neural Networks](https://mlsanity.com/paper/2409.11542/) (2024)
- [Diffusion-Based Particle-DETR for BEV Perception](https://mlsanity.com/paper/2312.11578/) (2023)
- [nuScenes Revisited: Progress and Challenges in Autonomous Driving](https://mlsanity.com/paper/2512.02448/) (2025)
- [Object criticality for safer navigation](https://mlsanity.com/paper/2406.10232/) (2024)
- [Scalable Offline Metrics for Autonomous Driving](https://mlsanity.com/paper/2510.08571/) (2025)
- [Energy-Efficient Autonomous Driving with Adaptive Perception and Robust Decision](https://mlsanity.com/paper/2510.25205/) (2025)
- [Physically Realistic Sequence-Level Adversarial Clothing for Robust Human-Detection Evasion](https://mlsanity.com/paper/2511.16020/) (2025)
- [STELLAR: Scaling 3D Perception Large Models for Autonomous Driving](https://mlsanity.com/paper/2605.20390/) (2026)
- [PointCHR: Point Cloud Analysis via Curvature-Aware Hyperbolic Rectification](https://mlsanity.com/paper/2607.24052/) (2026)

## Abstract (arXiv)

Prior work in 3D object detection evaluates models using offline metrics like average precision since closed-loop online evaluation on the downstream driving task is costly. However, it is unclear how indicative offline results are of driving performance. In this work, we perform the first empirical evaluation measuring how predictive different detection metrics are of driving performance when detectors are integrated into a full self-driving stack. We conduct extensive experiments on urban driving in the CARLA simulator using 16 object detection models. We find that the nuScenes Detection Score has a higher correlation to driving performance than the widely used average precision metric. In addition, our results call for caution on the exclusive reliance on the emerging class of `planner-centric' metrics.