# Goal-Conditioned Reinforcement Learning for Data-Driven Maritime Navigation

arXiv: https://arxiv.org/abs/2509.01838 · HTML: https://mlsanity.com/paper/2509.01838/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 44 references that are in the map.
- First posted on arXiv 1 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research](https://mlsanity.com/paper/1802.09464/) (2018)
- [A Closer Look at Invalid Action Masking in Policy Gradient Algorithms](https://mlsanity.com/paper/2006.14171/) (2020)
- [Goal-Conditioned Reinforcement Learning: Problems and Solutions](https://mlsanity.com/paper/2201.08299/) (2022)
- [TrAISformer -- A Transformer Network with Sparse Augmented Data Representation and Cross Entropy Loss for AIS-based Vessel Trajectory Prediction](https://mlsanity.com/paper/2109.03958/) (2021)
- [Multi-Path Long-Term Vessel Trajectories Forecasting with Probabilistic Feature Fusion for Problem Shifting](https://mlsanity.com/paper/2310.18948/) (2023)
- [Safe Multi-Agent Navigation guided by Goal-Conditioned Safe Reinforcement Learning](https://mlsanity.com/paper/2502.17813/) (2025)
- [Enhancing Maritime Trajectory Forecasting via H3 Index and Causal Language Modelling (CLM)](https://mlsanity.com/paper/2405.09596/) (2024)
- [Maritime Tracking Data Analysis and Integration with AISdb](https://mlsanity.com/paper/2407.08082/) (2024)
- [Physics-Informed Neural Networks for Vessel Trajectory Prediction: Learning Time-Discretized Kinematic Dynamics via Finite Differences](https://mlsanity.com/paper/2506.12029/) (2025)
- [Provable Traffic Rule Compliance in Safe Reinforcement Learning on the Open Sea](https://mlsanity.com/paper/2402.08502/) (2024)
- [AI on the Water: Applying DRL to Autonomous Vessel Navigation](https://mlsanity.com/paper/2310.14938/) (2023)
- [Modeling Maritime Transportation Behavior Using AIS Trajectories and Markovian Processes in the Gulf of St. Lawrence](https://mlsanity.com/paper/2506.00025/) (2025)

## Built on by (most cited first, 2 of 2)

- [World Models as Group Actions](https://mlsanity.com/paper/2605.24578/) (2026)
- [Does Latent Context Help? A Controlled Evaluation of Inverse Reinforcement Learning in Arctic Shipping](https://mlsanity.com/paper/2608.06105/) (2026)

## Abstract (arXiv)

Routing vessels through narrow and dynamic waterways is challenging due to changing environmental conditions and operational constraints. Existing vessel-routing studies typically fail to generalize across multiple origin-destination pairs and do not exploit large-scale, data-driven traffic graphs. In this paper, we propose a reinforcement learning solution for big maritime data that can learn to find a route across multiple origin-destination pairs while adapting to different hexagonal grid resolutions. Agents learn to select direction and speed under continuous observations in a multi-discrete action space. A reward function balances fuel efficiency, travel time, wind resistance, and route diversity, using an Automatic Identification System (AIS)-derived traffic graph with ERA5 wind fields. The approach is demonstrated in the Gulf of St. Lawrence, one of the largest estuaries in the world. We evaluate configurations that combine Proximal Policy Optimization with recurrent networks, invalid-action masking, and exploration strategies. Our experiments demonstrate that action masking yields a clear improvement in policy performance and that supplementing penalty-only feedback with positive shaping rewards produces additional gains.