# Criticality Metrics for Relevance Classification in Safety Evaluation of Object Detection in Automated Driving

arXiv: https://arxiv.org/abs/2512.15181 · HTML: https://mlsanity.com/paper/2512.15181/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 16 references that are in the map.
- First posted on arXiv 17 Dec 2025.

## Builds on (most cited first, 6 of 6)

- [On a Formal Model of Safe and Scalable Self-driving Cars](https://mlsanity.com/paper/1708.06374/) (2017)
- [DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Driving](https://mlsanity.com/paper/2304.01168/) (2023)
- [Criticality Metrics for Automated Driving: A Review and Suitability Analysis of the State of the Art](https://mlsanity.com/paper/2108.02403/) (2021)
- [LSM: A Comprehensive Metric for Assessing the Safety of Lane Detection Systems in Autonomous Driving](https://mlsanity.com/paper/2407.07740/) (2024)
- [A Comprehensive Safety Metric to Evaluate Perception in Autonomous Systems](https://mlsanity.com/paper/2512.14367/) (2025)
- [SURE-Val: Safe Urban Relevance Extension and Validation](https://mlsanity.com/paper/2308.02266/) (2023)

## Built on by (most cited first, 2 of 2)

- [Comprehensive Robustness Analysis of LiDAR-based 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/2607.02074/) (2026)
- [Scenario-Independent Criticality Assessment and Prediction for Vulnerable Road Users in Autonomous Driving](https://mlsanity.com/paper/2609.11947/) (2026)

## Abstract (arXiv)

Ensuring safety is the primary objective of automated driving, which necessitates a comprehensive and accurate perception of the environment. While numerous performance evaluation metrics exist for assessing perception capabilities, incorporating safety-specific metrics is essential to reliably evaluate object detection systems. A key component for safety evaluation is the ability to distinguish between relevant and non-relevant objects - a challenge addressed by criticality or relevance metrics. This paper presents the first in-depth analysis of criticality metrics for safety evaluation of object detection systems. Through a comprehensive review of existing literature, we identify and assess a range of applicable metrics. Their effectiveness is empirically validated using the DeepAccident dataset, which features a variety of safety-critical scenarios. To enhance evaluation accuracy, we propose two novel application strategies: bidirectional criticality rating and multi-metric aggregation. Our approach demonstrates up to a 100% improvement in terms of criticality classification accuracy, highlighting its potential to significantly advance the safety evaluation of object detection systems in automated vehicles.