Liability, Ethics, and Culture-Aware Behavior Specification using Rulebooks
Not on the map. Why?
Abstract (arXiv)
The behavior of self-driving cars must be compatible with an enormous set of conflicting and ambiguous objectives, from law, from ethics, from the local culture, and so on. This paper describes a new way to conveniently define the desired behavior for autonomous agents, which we use on the self-driving cars developed at nuTonomy. We define a "rulebook" as a pre-ordered set of "rules", each akin to a violation metric on the possible outcomes ("realizations"). The rules are partially ordered by priority. The semantics of a rulebook imposes a pre-order on the set of realizations. We study the compositional properties of the rulebooks, and we derive which operations we can allow on the rulebooks to preserve previously-introduced constraints. While we demonstrate the application of these techniques in the self-driving domain, the methods are domain-independent.
Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 Feb 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 7; 2022: 6; 2023: 3; 2024: 10; 2025: 5; 2026: 12.
43 papers.
Sort:
By area:
- 2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systemscited by 96
- 2022LaserMix for Semi-Supervised LiDAR Semantic Segmentationcited by 93
- 2022Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Drivingcited by 34
- 2024A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Drivingcited by 18
- 2021The Reasonable Crowd: Towards evidence-based and interpretable models of driving behaviorcited by 12
- 2021Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Reviewcited by 12
- 2023Fine-Tuning Language Models Using Formal Methods Feedbackcited by 12
- 2021Safety of the Intended Driving Behavior Using Rulebookscited by 9
- 2021Parallel and Multi-Objective Falsification with Scenic and VerifAIcited by 8
- 2021Hierarchical Potential-based Reward Shaping from Task Specificationscited by 7
- 2022Receding Horizon Planning with Rule Hierarchies for Autonomous Vehiclescited by 7
- 2024Dreaming to Assist: Learning to Align with Human Objectives for Shared Control in High-Speed Racingcited by 7
31 more
- 2024Informed Reinforcement Learning for Situation-Aware Traffic Rule Exceptionscited by 4
- 2022Hybrid Deep Reinforcement Learning and Planning for Safe and Comfortable Automated Drivingcited by 3
- 2023Multi-Predictor Fusion: Combining Learning-based and Rule-based Trajectory Predictorscited by 3
- 2024GENESIS-RL: GEnerating Natural Edge-cases with Systematic Integration of Safety considerations and Reinforcement Learningcited by 3
- 2021Immune Moral Models? Pro-Social Rule Breaking as a Moral Enhancement Approach for Ethical AIcited by 2
- 2022Quantification of Actual Road User Behavior on the Basis of Given Traffic Rulescited by 2
- 2024Provable Traffic Rule Compliance in Safe Reinforcement Learning on the Open Seacited by 2
- 2024RuleFuser: An Evidential Bayes Approach for Rule Injection in Imitation Learned Planners and Predictors for Robustness under Distribution Shiftscited by 2
- 2021Learning to falsify automated driving vehicles with prior knowledgecited by 1
- 2025Predictive Traffic Rule Compliance using Reinforcement Learningcited by 1
- 2025Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Drivingcited by 1
- 2026PoSafeNet: Safe Learning with Poset-Structured Neural Netscited by 1
- 2026ScenicRules: An Autonomous Driving Benchmark with Multi-Objective Specifications and Abstract Scenarioscited by 1
- 2026RECTOR: Priority-Aware Rule-Based Reranking for Compliance-Aware Autonomous Driving Trajectory Selectioncited by 1
- 2026Steering Generative Robot Policies with Lexicographic Preferencescited by 1
- 2022Safe Policy Improvement in Constrained Markov Decision Processescited by 0
- 2023Specification-Guided Data Aggregation for Semantically Aware Imitation Learningcited by 0
- 2024Virtue Ethics For Ethically Tunable Robotic Assistantscited by 0
- 2024An Ontology-based Approach Towards Traceable Behavior Specifications in Automated Drivingcited by 0
- 2024Incorporating System-level Safety Requirements in Perception Models via Reinforcement Learningcited by 0
- 2025Cross-cultural Deployment of Autonomous Vehicles Using Data-light Inverse Reinforcement Learningcited by 0
- 2025Integrating Legal and Logical Specifications in Perception, Prediction, and Planning for Automated Driving: A Survey of Methodscited by 0
- 2025Multi-Objective Search: Algorithms, Applications, and Emerging Directionscited by 0
- 2026Beyond Scalar Rewards: Distributional Reinforcement Learning with Preordered Objectives for Safe and Reliable Autonomous Drivingcited by 0
- 2026CARVE: Certified Affordable Repair of Vetoed Maneuvers via Envelopes for Interactive Drivingcited by 0
- 2026CARVE-Q: Quantum-Proposed, Classically Certified Interactive Driving Repaircited by 0
- 2026Real-Time Rulebook-Aware Nonlinear MPC for Autonomous Driving with Priority-Biased Tiered Slackscited by 0
- 2026Approximate Multi-Objective Search Under Rulebookscited by 0
- 2026Structured Four-Stage Legal Translation: From Natural-Language Traffic Rules to PROLOGcited by 0
- 2026Commonsense-Grounded Path Planning from Abstract Instructionscited by 0
- 2026MakSafe: A Spatiotemporal Semantic Framework for Formal Specification and Compliance Checking of Road Vehicle Safety Rulescited by 0
- 2026MakSafe: A Spatiotemporal Semantic Framework for Formal Specification and Compliance Checking of Road Vehicle Safety Rules14 Oct 2026
- 2026Commonsense-Grounded Path Planning from Abstract Instructions19 Sep 2026
- 2026Structured Four-Stage Legal Translation: From Natural-Language Traffic Rules to PROLOG17 Sep 2026
- 2026Steering Generative Robot Policies with Lexicographic Preferences14 Sep 2026
- 2026Approximate Multi-Objective Search Under Rulebooks5 Aug 2026
- 2026Real-Time Rulebook-Aware Nonlinear MPC for Autonomous Driving with Priority-Biased Tiered Slacks13 Jul 2026
- 2026CARVE-Q: Quantum-Proposed, Classically Certified Interactive Driving Repair3 Jun 2026
- 2026CARVE: Certified Affordable Repair of Vetoed Maneuvers via Envelopes for Interactive Driving31 May 2026
- 2026RECTOR: Priority-Aware Rule-Based Reranking for Compliance-Aware Autonomous Driving Trajectory Selection24 May 2026
- 2026Beyond Scalar Rewards: Distributional Reinforcement Learning with Preordered Objectives for Safe and Reliable Autonomous Driving6 Mar 2026
- 2026ScenicRules: An Autonomous Driving Benchmark with Multi-Objective Specifications and Abstract Scenarios17 Feb 2026
- 2026PoSafeNet: Safe Learning with Poset-Structured Neural Nets29 Jan 2026
31 more
- 2025Multi-Objective Search: Algorithms, Applications, and Emerging Directions29 Oct 2025
- 2025Integrating Legal and Logical Specifications in Perception, Prediction, and Planning for Automated Driving: A Survey of Methods29 Oct 2025
- 2025Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Driving10 May 2025
- 2025Cross-cultural Deployment of Autonomous Vehicles Using Data-light Inverse Reinforcement Learning15 Apr 2025
- 2025Predictive Traffic Rule Compliance using Reinforcement Learning29 Mar 2025
- 2024Incorporating System-level Safety Requirements in Perception Models via Reinforcement Learning4 Dec 2024
- 2024Dreaming to Assist: Learning to Align with Human Objectives for Shared Control in High-Speed Racing14 Oct 2024
- 2024An Ontology-based Approach Towards Traceable Behavior Specifications in Automated Driving10 Sep 2024
- 2024Virtue Ethics For Ethically Tunable Robotic Assistants23 Jul 2024
- 2024RuleFuser: An Evidential Bayes Approach for Rule Injection in Imitation Learned Planners and Predictors for Robustness under Distribution Shifts18 May 2024
- 2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems10 May 2024
- 2024A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving12 Apr 2024
- 2024GENESIS-RL: GEnerating Natural Edge-cases with Systematic Integration of Safety considerations and Reinforcement Learning27 Mar 2024
- 2024Provable Traffic Rule Compliance in Safe Reinforcement Learning on the Open Sea13 Feb 2024
- 2024Informed Reinforcement Learning for Situation-Aware Traffic Rule Exceptions6 Feb 2024
- 2023Fine-Tuning Language Models Using Formal Methods Feedback27 Oct 2023
- 2023Multi-Predictor Fusion: Combining Learning-based and Rule-based Trajectory Predictors3 Jul 2023
- 2023Specification-Guided Data Aggregation for Semantically Aware Imitation Learning29 Mar 2023
- 2022Hybrid Deep Reinforcement Learning and Planning for Safe and Comfortable Automated Driving30 Dec 2022
- 2022Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving17 Dec 2022
- 2022Receding Horizon Planning with Rule Hierarchies for Autonomous Vehicles6 Dec 2022
- 2022Safe Policy Improvement in Constrained Markov Decision Processes20 Oct 2022
- 2022LaserMix for Semi-Supervised LiDAR Semantic Segmentation30 Jun 2022
- 2022Quantification of Actual Road User Behavior on the Basis of Given Traffic Rules7 Feb 2022
- 2021Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Review16 Oct 2021
- 2021Hierarchical Potential-based Reward Shaping from Task Specifications6 Oct 2021
- 2021The Reasonable Crowd: Towards evidence-based and interpretable models of driving behavior28 Jul 2021
- 2021Parallel and Multi-Objective Falsification with Scenic and VerifAI9 Jul 2021
- 2021Immune Moral Models? Pro-Social Rule Breaking as a Moral Enhancement Approach for Ethical AI17 Jun 2021
- 2021Safety of the Intended Driving Behavior Using Rulebooks10 May 2021
- 2021Learning to falsify automated driving vehicles with prior knowledge25 Jan 2021
- 2022Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Drivingcited by 34
- 2024A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Drivingcited by 18
- 2022Receding Horizon Planning with Rule Hierarchies for Autonomous Vehiclescited by 7
- 2022Hybrid Deep Reinforcement Learning and Planning for Safe and Comfortable Automated Drivingcited by 3
- 2025Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Drivingcited by 1
- 2025Cross-cultural Deployment of Autonomous Vehicles Using Data-light Inverse Reinforcement Learningcited by 0
- 2026Beyond Scalar Rewards: Distributional Reinforcement Learning with Preordered Objectives for Safe and Reliable Autonomous Drivingcited by 0
- 2026CARVE: Certified Affordable Repair of Vetoed Maneuvers via Envelopes for Interactive Drivingcited by 0
- 2026Real-Time Rulebook-Aware Nonlinear MPC for Autonomous Driving with Priority-Biased Tiered Slackscited by 0
- 2026Real-Time Rulebook-Aware Nonlinear MPC for Autonomous Driving with Priority-Biased Tiered Slacks13 Jul 2026
- 2026CARVE: Certified Affordable Repair of Vetoed Maneuvers via Envelopes for Interactive Driving31 May 2026
- 2026Beyond Scalar Rewards: Distributional Reinforcement Learning with Preordered Objectives for Safe and Reliable Autonomous Driving6 Mar 2026
- 2025Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Driving10 May 2025
- 2025Cross-cultural Deployment of Autonomous Vehicles Using Data-light Inverse Reinforcement Learning15 Apr 2025
- 2024A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving12 Apr 2024
- 2022Hybrid Deep Reinforcement Learning and Planning for Safe and Comfortable Automated Driving30 Dec 2022
- 2022Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving17 Dec 2022
- 2022Receding Horizon Planning with Rule Hierarchies for Autonomous Vehicles6 Dec 2022
- 2024Informed Reinforcement Learning for Situation-Aware Traffic Rule Exceptionscited by 4
- 2024GENESIS-RL: GEnerating Natural Edge-cases with Systematic Integration of Safety considerations and Reinforcement Learningcited by 3
- 2024Provable Traffic Rule Compliance in Safe Reinforcement Learning on the Open Seacited by 2
- 2025Predictive Traffic Rule Compliance using Reinforcement Learningcited by 1
- 2024Incorporating System-level Safety Requirements in Perception Models via Reinforcement Learningcited by 0
- 2025Predictive Traffic Rule Compliance using Reinforcement Learning29 Mar 2025
- 2024Incorporating System-level Safety Requirements in Perception Models via Reinforcement Learning4 Dec 2024
- 2024GENESIS-RL: GEnerating Natural Edge-cases with Systematic Integration of Safety considerations and Reinforcement Learning27 Mar 2024
- 2024Provable Traffic Rule Compliance in Safe Reinforcement Learning on the Open Sea13 Feb 2024
- 2024Informed Reinforcement Learning for Situation-Aware Traffic Rule Exceptions6 Feb 2024
- 2022LaserMix for Semi-Supervised LiDAR Semantic Segmentationcited by 93
- 2021The Reasonable Crowd: Towards evidence-based and interpretable models of driving behaviorcited by 12
- 2021Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Reviewcited by 12
- 2021Safety of the Intended Driving Behavior Using Rulebookscited by 9
- 2021Parallel and Multi-Objective Falsification with Scenic and VerifAIcited by 8
- 2021Hierarchical Potential-based Reward Shaping from Task Specificationscited by 7
- 2023Multi-Predictor Fusion: Combining Learning-based and Rule-based Trajectory Predictorscited by 3
- 2021Immune Moral Models? Pro-Social Rule Breaking as a Moral Enhancement Approach for Ethical AIcited by 2
- 2022Quantification of Actual Road User Behavior on the Basis of Given Traffic Rulescited by 2
- 2021Learning to falsify automated driving vehicles with prior knowledgecited by 1
- 2026PoSafeNet: Safe Learning with Poset-Structured Neural Netscited by 1
- 2026ScenicRules: An Autonomous Driving Benchmark with Multi-Objective Specifications and Abstract Scenarioscited by 1
12 most cited of 23 All 23 (JSON)
- 2026MakSafe: A Spatiotemporal Semantic Framework for Formal Specification and Compliance Checking of Road Vehicle Safety Rules14 Oct 2026
- 2026Commonsense-Grounded Path Planning from Abstract Instructions19 Sep 2026
- 2026Structured Four-Stage Legal Translation: From Natural-Language Traffic Rules to PROLOG17 Sep 2026
- 2026Approximate Multi-Objective Search Under Rulebooks5 Aug 2026
- 2026CARVE-Q: Quantum-Proposed, Classically Certified Interactive Driving Repair3 Jun 2026
- 2026RECTOR: Priority-Aware Rule-Based Reranking for Compliance-Aware Autonomous Driving Trajectory Selection24 May 2026
- 2026ScenicRules: An Autonomous Driving Benchmark with Multi-Objective Specifications and Abstract Scenarios17 Feb 2026
- 2026PoSafeNet: Safe Learning with Poset-Structured Neural Nets29 Jan 2026
- 2025Multi-Objective Search: Algorithms, Applications, and Emerging Directions29 Oct 2025
- 2025Integrating Legal and Logical Specifications in Perception, Prediction, and Planning for Automated Driving: A Survey of Methods29 Oct 2025
- 2024An Ontology-based Approach Towards Traceable Behavior Specifications in Automated Driving10 Sep 2024
- 2024Virtue Ethics For Ethically Tunable Robotic Assistants23 Jul 2024
12 newest of 23 All 23 (JSON)