mlsanityData to 7 Oct 2026

Evaluating Object (mis)Detection from a Safety and Reliability Perspective: Discussion and Measures

arXiv 2203.02205 ↗First posted 4 Mar 2022cs.LG cs.AI cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

We argue that object detectors in the safety critical domain should prioritize detection of objects that are most likely to interfere with the actions of the autonomous actor. Especially, this applies to objects that can impact the actor's safety and reliability. To quantify the impact of object (mis)detection on safety and reliability in the context of autonomous driving, we propose new object detection measures that reward the correct identification of objects that are most dangerous and most likely to affect driving decisions. To achieve this, we build an object criticality model to reward the detection of the objects based on proximity, orientation, and relative velocity with respect to the subject vehicle. Then, we apply our model on the recent autonomous driving dataset nuScenes, and we compare nine object detectors. Results show that, in several settings, object detectors that perform best according to the nuScenes ranking are not the preferable ones when the focus is shifted on safety and reliability.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 24 of its 48 references that are in the map. First posted on arXiv 4 Mar 2022.

Builds on

24 of 48 references.

References · 24

  1. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  5. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  6. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  7. 2016Concrete Problems in AI Safetycited by 1,582
  8. 2017Deformable Convolutional Networkscited by 1,481
  9. 2020Designing Network Design Spacescited by 954
  10. 2018Object Detection with Deep Learning: A Reviewcited by 524
  11. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  12. 2020Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlationcited by 420
12 more
  1. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 401
  2. 2018Deep Learning for Generic Object Detection: A Surveycited by 392
  3. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
  4. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 190
  5. 2018Bounding Box Regression with Uncertainty for Accurate Object Detectioncited by 143
  6. 2019A General Framework for Uncertainty Estimation in Deep Learningcited by 119
  7. 2019A Survey of Deep Learning-based Object Detectioncited by 106
  8. 2018Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detectioncited by 82
  9. 2020SSN: Shape Signature Networks for Multi-class Object Detection from Point Cloudscited by 74
  10. 2016On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Productscited by 52
  11. 2021Risk Ranked Recall: Collision Safety Metric for Object Detection Systems in Autonomous Vehiclescited by 10
  12. 2020Safety-Aware Hardening of 3D Object Detection Neural Network Systemscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 2; 2026: 3.

8 papers.

  1. 2024A Flow-based Credibility Metric for Safety-critical Pedestrian Detectioncited by 2
  2. 2024Object criticality for safer navigationcited by 1
  3. 2023Checklist to Define the Identification of TP, FP, and FN Object Detections in Automated Drivingcited by 0
  4. 2025Dynamic Classification: Leveraging Self-Supervised Classification to Enhance Prediction Performancecited by 0
  5. 2025Rethink 3D Object Detection from Physical Worldcited by 0
  6. 2026RoAD Benchmark: How LiDAR Models Fail under Coupled Domain Shifts and Label Evolutioncited by 0
  7. 2026Effort-Based Criticality Metrics for Evaluating 3D Perception Errors in Autonomous Drivingcited by 0
  8. 2026Detecting Backdoors in Object Detection via Pre-NMS Prediction Distribution Shiftcited by 0