# Benchmarks for Retrospective Automated Driving System Crash Rate Analysis Using Police-Reported Crash Data

arXiv: https://arxiv.org/abs/2312.13228 · HTML: https://mlsanity.com/paper/2312.13228/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Dec 2023.

## Built on by (most cited first, 2 of 2)

- [The Case for Negative Data: From Crash Reports to Counterfactuals for Reasonable Driving](https://mlsanity.com/paper/2509.18626/) (2025)
- [Credibility-Weighted Pricing of Autonomous Vehicle Liability Under Operational Design Domain Shift](https://mlsanity.com/paper/2606.17451/) (2026)

## Abstract (arXiv)

With fully automated driving systems (ADS; SAE level 4) ride-hailing services expanding in the US, we are now approaching an inflection point, where the process of retrospectively evaluating ADS safety impact can start to yield statistically credible conclusions. An ADS safety impact measurement requires a comparison to a "benchmark" crash rate. This study aims to address, update, and extend the existing literature by leveraging police-reported crashes to generate human crash rates for multiple geographic areas with current ADS deployments. All of the data leveraged is publicly accessible, and the benchmark determination methodology is intended to be repeatable and transparent. Generating a benchmark that is comparable to ADS crash data is associated with certain challenges, including data selection, handling underreporting and reporting thresholds, identifying the population of drivers and vehicles to compare against, choosing an appropriate severity level to assess, and matching crash and mileage exposure data. Consequently, we identify essential steps when generating benchmarks, and present our analyses amongst a backdrop of existing ADS benchmark literature. One analysis presented is the usage of established underreporting correction methodology to publicly available human driver police-reported data to improve comparability to publicly available ADS crash data. We also identify important dependencies in controlling for geographic region, road type, and vehicle type, and show how failing to control for these features can bias results. This body of work aims to contribute to the ability of the community - researchers, regulators, industry, and experts - to reach consensus on how to estimate accurate benchmarks.