# A Safety-Adapted Loss for Pedestrian Detection in Automated Driving

arXiv: https://arxiv.org/abs/2402.02986 · HTML: https://mlsanity.com/paper/2402.02986/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 15 of 15)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [A Survey of Modern Deep Learning based Object Detection Models](https://mlsanity.com/paper/2104.11892/) (2021)
- [Learning to Evaluate Perception Models Using Planner-Centric Metrics](https://mlsanity.com/paper/2004.08745/) (2020)
- [Equalized Focal Loss for Dense Long-Tailed Object Detection](https://mlsanity.com/paper/2201.02593/) (2022)
- [Safety Concerns and Mitigation Approaches Regarding the Use of Deep Learning in Safety-Critical Perception Tasks](https://mlsanity.com/paper/2001.08001/) (2020)
- [Binary TTC: A Temporal Geofence for Autonomous Navigation](https://mlsanity.com/paper/2101.04777/) (2021)
- [Which Samples Should be Learned First: Easy or Hard?](https://mlsanity.com/paper/2110.05481/) (2021)
- [Risk Ranked Recall: Collision Safety Metric for Object Detection Systems in Autonomous Vehicles](https://mlsanity.com/paper/2106.04146/) (2021)
- [Safety-Aware Hardening of 3D Object Detection Neural Network Systems](https://mlsanity.com/paper/2003.11242/) (2020)
- [Interaction-Dynamics-Aware Perception Zones for Obstacle Detection Safety Evaluation](https://mlsanity.com/paper/2206.12471/) (2022)
- [USC: Uncompromising Spatial Constraints for Safety-Oriented 3D Object Detectors in Autonomous Driving](https://mlsanity.com/paper/2209.10368/) (2022)
- [Refining Obstacle Perception Safety Zones via Maneuver-Based Decomposition](https://mlsanity.com/paper/2308.06337/) (2023)

## Built on by (most cited first, 2 of 2)

- [Safety-Aligned 3D Object Detection: Single-Vehicle, Cooperative, and End-to-End Perspectives](https://mlsanity.com/paper/2604.03325/) (2026)
- [Can Attack Difficulty Be Characterized Before Optimization? A Study of Pre-optimization Difficulty in Person-Vanishing Attacks](https://mlsanity.com/paper/2609.34501/) (2026)

## Abstract (arXiv)

In safety-critical domains like automated driving (AD), errors by the object detector may endanger pedestrians and other vulnerable road users (VRU). As common evaluation metrics are not an adequate safety indicator, recent works employ approaches to identify safety-critical VRU and back-annotate the risk to the object detector. However, those approaches do not consider the safety factor in the deep neural network (DNN) training process. Thus, state-of-the-art DNN penalizes all misdetections equally irrespective of their criticality. Subsequently, to mitigate the occurrence of critical failure cases, i.e., false negatives, a safety-aware training strategy might be required to enhance the detection performance for critical pedestrians. In this paper, we propose a novel safety-aware loss variation that leverages the estimated per-pedestrian criticality scores during training. We exploit the reachability set-based time-to-collision (TTC-RSB) metric from the motion domain along with distance information to account for the worst-case threat quantifying the criticality. Our evaluation results using RetinaNet and FCOS on the nuScenes dataset demonstrate that training the models with our safety-aware loss function mitigates the misdetection of critical pedestrians without sacrificing performance for the general case, i.e., pedestrians outside the safety-critical zone.