# Falsification-driven reinforcement learning for maritime motion planning

arXiv: https://arxiv.org/abs/2510.06970 · HTML: https://mlsanity.com/paper/2510.06970/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 76 references that are in the map.
- First posted on arXiv 8 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective](https://mlsanity.com/paper/2202.02215/) (2022)
- [A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems](https://mlsanity.com/paper/2005.02979/) (2020)
- ["Curse of rarity" for autonomous vehicles](https://mlsanity.com/paper/2207.02749/) (2022)
- [Risk Averse Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1904.00511/) (2019)
- [evosax: JAX-based Evolution Strategies](https://mlsanity.com/paper/2212.04180/) (2022)
- [Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarking](https://mlsanity.com/paper/2205.06750/) (2022)
- [Falsification of Cyber-Physical Systems Using Deep Reinforcement Learning](https://mlsanity.com/paper/1805.00200/) (2018)
- [COLREG-Compliant Collision Avoidance for Unmanned Surface Vehicle using Deep Reinforcement Learning](https://mlsanity.com/paper/2006.09540/) (2020)
- [cmaes: A Simple yet Practical Python Library for CMA-ES](https://mlsanity.com/paper/2402.01373/) (2024)
- [Gray-box Adversarial Testing for Control Systems with Machine Learning Component](https://mlsanity.com/paper/1812.11958/) (2018)
- [Falsification of Cyber-Physical Systems with Robustness-Guided Black-Box Checking](https://mlsanity.com/paper/2005.02126/) (2020)
- [Falsification-Based Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/2007.00691/) (2020)
- [Falsification of Learning-Based Controllers through Multi-Fidelity Bayesian Optimization](https://mlsanity.com/paper/2212.14118/) (2022)
- [RTAMT -- Runtime Robustness Monitors with Application to CPS and Robotics](https://mlsanity.com/paper/2501.18608/) (2025)
- [Falsification of Cyber-Physical Systems using Bayesian Optimization](https://mlsanity.com/paper/2209.06735/) (2022)
- [Provable Traffic Rule Compliance in Safe Reinforcement Learning on the Open Sea](https://mlsanity.com/paper/2402.08502/) (2024)
- [Autonomy for Ferries and Harbour Buses: a Collision Avoidance Perspective](https://mlsanity.com/paper/2301.02711/) (2023)

## Abstract (arXiv)

Compliance with maritime traffic rules is essential for the safe operation of autonomous vessels, yet training reinforcement learning (RL) agents to adhere to them is challenging. The behavior of RL agents is shaped by the training scenarios they encounter, but creating scenarios that capture the complexity of maritime navigation is non-trivial, and real-world data alone is insufficient. To address this, we propose a falsification-driven RL approach that generates adversarial training scenarios in which the vessel under test violates maritime traffic rules, which are expressed as signal temporal logic specifications. Our experiments on open-sea navigation with two vessels demonstrate that the proposed approach provides more relevant training scenarios and achieves more consistent rule compliance.