# ScenicRules: An Autonomous Driving Benchmark with Multi-Objective Specifications and Abstract Scenarios

arXiv: https://arxiv.org/abs/2602.16073 · HTML: https://mlsanity.com/paper/2602.16073/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 67 references that are in the map.
- First posted on arXiv 17 Feb 2026.

## Builds on (most cited first, 27 of 27)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles](https://mlsanity.com/paper/2106.11810/) (2021)
- [NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking](https://mlsanity.com/paper/2406.15349/) (2024)
- [MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning](https://mlsanity.com/paper/2109.12674/) (2021)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps](https://mlsanity.com/paper/1910.03088/) (2019)
- [Pseudo-Simulation for Autonomous Driving](https://mlsanity.com/paper/2506.04218/) (2025)
- [On a Formal Model of Safe and Scalable Self-driving Cars](https://mlsanity.com/paper/1708.06374/) (2017)
- [Towards learning-based planning:The nuPlan benchmark for real-world autonomous driving](https://mlsanity.com/paper/2403.04133/) (2024)
- [Scenic: A Language for Scenario Specification and Scene Generation](https://mlsanity.com/paper/1809.09310/) (2018)
- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [Liability, Ethics, and Culture-Aware Behavior Specification using Rulebooks](https://mlsanity.com/paper/1902.09355/) (2019)
- [Scenic: A Language for Scenario Specification and Data Generation](https://mlsanity.com/paper/2010.06580/) (2020)
- [A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advances](https://mlsanity.com/paper/2505.17799/) (2025)
- [The Reasonable Crowd: Towards evidence-based and interpretable models of driving behavior](https://mlsanity.com/paper/2107.13507/) (2021)
- [ScenicNL: Generating Probabilistic Scenario Programs from Natural Language](https://mlsanity.com/paper/2405.03709/) (2024)
- [Safety of the Intended Driving Behavior Using Rulebooks](https://mlsanity.com/paper/2105.04472/) (2021)
- [Parallel and Multi-Objective Falsification with Scenic and VerifAI](https://mlsanity.com/paper/2107.04164/) (2021)
- [Addressing the IEEE AV Test Challenge with Scenic and VerifAI](https://mlsanity.com/paper/2108.13796/) (2021)
- [M3CAD: Towards Generic Cooperative Autonomous Driving Benchmark](https://mlsanity.com/paper/2505.06746/) (2025)
- [SceNDD: A Scenario-based Naturalistic Driving Dataset](https://mlsanity.com/paper/2212.12436/) (2022)
- [scenario.center: Methods from Real-world Data to a Scenario Database](https://mlsanity.com/paper/2404.02561/) (2024)

## Built on by (most cited first, 1 of 1)

- [Revealing Interpretable Failure Modes of VLMs](https://mlsanity.com/paper/2605.12674/) (2026)

## Abstract (arXiv)

Developing autonomous driving systems for complex traffic environments requires balancing multiple objectives, such as avoiding collisions, obeying traffic rules, and making efficient progress. In many situations, these objectives cannot be satisfied simultaneously, and explicit priority relations naturally arise. Also, driving rules require context, so it is important to formally model the environment scenarios within which such rules apply. Existing benchmarks for evaluating autonomous vehicles lack such combinations of multi-objective prioritized rules and formal environment models. In this work, we introduce ScenicRules, a benchmark for evaluating autonomous driving systems in stochastic environments under prioritized multi-objective specifications. We first formalize a diverse set of objectives to serve as quantitative evaluation metrics. Next, we design a Hierarchical Rulebook framework that encodes multiple objectives and their priority relations in an interpretable and adaptable manner. We then construct a compact yet representative collection of scenarios spanning diverse driving contexts and near-accident situations, formally modeled in the Scenic language. Experimental results show that our formalized objectives and Hierarchical Rulebooks align well with human driving judgments and that our benchmark effectively exposes agent failures with respect to the prioritized objectives. Our benchmark can be accessed at https://github.com/BerkeleyLearnVerify/ScenicRules/.