# Provable Traffic Rule Compliance in Safe Reinforcement Learning on the Open Sea

arXiv: https://arxiv.org/abs/2402.08502 · HTML: https://mlsanity.com/paper/2402.08502/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 17 of its 80 references that are in the map.
- First posted on arXiv 13 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 17 of 17)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Liability, Ethics, and Culture-Aware Behavior Specification using Rulebooks](https://mlsanity.com/paper/1902.09355/) (2019)
- [Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarking](https://mlsanity.com/paper/2205.06750/) (2022)
- [Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components](https://mlsanity.com/paper/1804.06760/) (2018)
- [Provably Safe Reinforcement Learning via Action Projection using Reachability Analysis and Polynomial Zonotopes](https://mlsanity.com/paper/2210.10691/) (2022)
- [Computationally Efficient Safe Reinforcement Learning for Power Systems](https://mlsanity.com/paper/2110.10333/) (2021)
- [COLREG-Compliant Collision Avoidance for Unmanned Surface Vehicle using Deep Reinforcement Learning](https://mlsanity.com/paper/2006.09540/) (2020)
- [Verifiably Safe Off-Model Reinforcement Learning](https://mlsanity.com/paper/1902.05632/) (2019)
- [Formalizing Traffic Rules for Machine Interpretability](https://mlsanity.com/paper/2007.00330/) (2020)
- [A Survey of Deep Reinforcement Learning Algorithms for Motion Planning and Control of Autonomous Vehicles](https://mlsanity.com/paper/2105.14218/) (2021)
- [Risk-based implementation of COLREGs for autonomous surface vehicles using deep reinforcement learning](https://mlsanity.com/paper/2112.00115/) (2021)
- [Fully-Automated Verification of Linear Systems Using Inner- and Outer-Approximations of Reachable Sets](https://mlsanity.com/paper/2209.09321/) (2022)
- [The Branching-Course MPC Algorithm for Maritime Collision Avoidance](https://mlsanity.com/paper/1907.00039/) (2019)
- [A Survey of Recent Machine Learning Solutions for Ship Collision Avoidance and Mission Planning](https://mlsanity.com/paper/2207.02767/) (2022)
- [COLREGs-Informed RRT* for Collision Avoidance of Marine Crafts](https://mlsanity.com/paper/2103.14426/) (2021)

## Built on by (most cited first, 2 of 2)

- [Goal-Conditioned Reinforcement Learning for Data-Driven Maritime Navigation](https://mlsanity.com/paper/2509.01838/) (2025)
- [Falsification-driven reinforcement learning for maritime motion planning](https://mlsanity.com/paper/2510.06970/) (2025)

## Abstract (arXiv)

For safe operation, autonomous vehicles have to obey traffic rules that are set forth in legal documents formulated in natural language. Temporal logic is a suitable concept to formalize such traffic rules. Still, temporal logic rules often result in constraints that are hard to solve using optimization-based motion planners. Reinforcement learning (RL) is a promising method to find motion plans for autonomous vehicles. However, vanilla RL algorithms are based on random exploration and do not automatically comply with traffic rules. Our approach accomplishes guaranteed rule-compliance by integrating temporal logic specifications into RL. Specifically, we consider the application of vessels on the open sea, which must adhere to the Convention on the International Regulations for Preventing Collisions at Sea (COLREGS). To efficiently synthesize rule-compliant actions, we combine predicates based on set-based prediction with a statechart representing our formalized rules and their priorities. Action masking then restricts the RL agent to this set of verified rule-compliant actions. In numerical evaluations on critical maritime traffic situations, our agent always complies with the formalized legal rules and never collides while achieving a high goal-reaching rate during training and deployment. In contrast, vanilla and traffic rule-informed RL agents frequently violate traffic rules and collide even after training.