# A Counterfactual Safety Margin Perspective on the Scoring of Autonomous Vehicles' Riskiness

arXiv: https://arxiv.org/abs/2308.01050 · HTML: https://mlsanity.com/paper/2308.01050/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial scenarios and safety testing, one of 9 directions in End-to-end autonomous driving.
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 2 Aug 2023.

## Placed in

- End-to-end autonomous driving / [Adversarial scenarios and safety testing](https://mlsanity.com/driving/adversarial-scenarios-and-safety-testing/)

## Builds on (most cited first, 11 of 11)

- [Guided Conditional Diffusion for Controllable Traffic Simulation](https://mlsanity.com/paper/2210.17366/) (2022)
- [Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior](https://mlsanity.com/paper/2112.05077/) (2021)
- [Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulation](https://mlsanity.com/paper/2205.03195/) (2022)
- [Counterfactual Risk Assessments, Evaluation, and Fairness](https://mlsanity.com/paper/1909.00066/) (2019)
- [Criticality Metrics for Automated Driving: A Review and Suitability Analysis of the State of the Art](https://mlsanity.com/paper/2108.02403/) (2021)
- [Waymo Public Road Safety Performance Data](https://mlsanity.com/paper/2011.00038/) (2020)
- [Eventual Discounting Temporal Logic Counterfactual Experience Replay](https://mlsanity.com/paper/2303.02135/) (2023)
- [Building a Credible Case for Safety: Waymo's Approach for the Determination of Absence of Unreasonable Risk](https://mlsanity.com/paper/2306.01917/) (2023)
- [Discovering Avoidable Planner Failures of Autonomous Vehicles using Counterfactual Analysis in Behaviorally Diverse Simulation](https://mlsanity.com/paper/2011.11991/) (2020)
- [Towards Data-Driven Synthesis of Autonomous Vehicle Safety Concepts](https://mlsanity.com/paper/2107.14412/) (2021)
- [Formal Estimation of Collision Risks for Autonomous Vehicles: A Compositional Data-Driven Approach](https://mlsanity.com/paper/2112.07187/) (2021)

## Built on by (most cited first, 3 of 3)

- [CounterScene: Counterfactual Causal Reasoning in Generative World Models for Safety-Critical Closed-Loop Evaluation](https://mlsanity.com/paper/2603.21104/) (2026)
- [The Case for Negative Data: From Crash Reports to Counterfactuals for Reasonable Driving](https://mlsanity.com/paper/2509.18626/) (2025)
- [Robusto-1 Dataset: Comparing Humans and VLMs on real out-of-distribution Autonomous Driving VQA from Peru](https://mlsanity.com/paper/2503.07587/) (2025)

## Abstract (arXiv)

Autonomous Vehicles (AVs) promise a range of societal advantages, including broader access to mobility, reduced road accidents, and enhanced transportation efficiency. However, evaluating the risks linked to AVs is complex due to limited historical data and the swift progression of technology. This paper presents a data-driven framework for assessing the risk of different AVs' behaviors in various operational design domains (ODDs), based on counterfactual simulations of "misbehaving" road users. We propose the notion of counterfactual safety margin, which represents the minimum deviation from nominal behavior that could cause a collision. This methodology not only pinpoints the most critical scenarios but also quantifies the (relative) risk's frequency and severity concerning AVs. Importantly, we show that our approach is applicable even when the AV's behavioral policy remains undisclosed, through worst- and best-case analyses, benefiting external entities like regulators and risk evaluators. Our experimental outcomes demonstrate the correlation between the safety margin, the quality of the driving policy, and the ODD, shedding light on the relative risks of different AV providers. Overall, this work contributes to the safety assessment of AVs and addresses legislative and insurance concerns surrounding this burgeoning technology.