Goal-Conditioned Reinforcement Learning for Data-Driven Maritime Navigation
On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control
Abstract (arXiv)
Routing vessels through narrow and dynamic waterways is challenging due to changing environmental conditions and operational constraints. Existing vessel-routing studies typically fail to generalize across multiple origin-destination pairs and do not exploit large-scale, data-driven traffic graphs. In this paper, we propose a reinforcement learning solution for big maritime data that can learn to find a route across multiple origin-destination pairs while adapting to different hexagonal grid resolutions. Agents learn to select direction and speed under continuous observations in a multi-discrete action space. A reward function balances fuel efficiency, travel time, wind resistance, and route diversity, using an Automatic Identification System (AIS)-derived traffic graph with ERA5 wind fields. The approach is demonstrated in the Gulf of St. Lawrence, one of the largest estuaries in the world. We evaluate configurations that combine Proximal Policy Optimization with recurrent networks, invalid-action masking, and exploration strategies. Our experiments demonstrate that action masking yields a clear improvement in policy performance and that supplementing penalty-only feedback with positive shaping rewards produces additional gains.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 44 references that are in the map. First posted on arXiv 1 Sep 2025.
Builds on
14 of 44 references.
References · 14
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2018Exploration by Random Network Distillationcited by 993
- 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
- 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 171
- 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
- 2021TrAISformer -- A Transformer Network with Sparse Augmented Data Representation and Cross Entropy Loss for AIS-based Vessel Trajectory Predictioncited by 20
- 2023Multi-Path Long-Term Vessel Trajectories Forecasting with Probabilistic Feature Fusion for Problem Shiftingcited by 12
- 2025Safe Multi-Agent Navigation guided by Goal-Conditioned Safe Reinforcement Learningcited by 6
- 2024Enhancing Maritime Trajectory Forecasting via H3 Index and Causal Language Modelling (CLM)cited by 3
- 2024Maritime Tracking Data Analysis and Integration with AISdbcited by 3
- 2025Physics-Informed Neural Networks for Vessel Trajectory Prediction: Learning Time-Discretized Kinematic Dynamics via Finite Differencescited by 3
- 2024Provable Traffic Rule Compliance in Safe Reinforcement Learning on the Open Seacited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Critic-Free Deep Reinforcement Learning for Maritime Coverage Path Planning on Irregular Hexagonal Grids
- 2026Physics-informed offline reinforcement learning eliminates catastrophic fuel waste in maritime routing
- 2026Realistic Curriculum Reinforcement Learning for Autonomous and Sustainable Marine Vessel Navigation
- 2024Evaluating Robustness of Reinforcement Learning Algorithms for Autonomous Shipping
- 2025Long-Term Mapping of the Douro River Plume with Multi-Agent Reinforcement Learning
- 2025Falsification-driven reinforcement learning for maritime motion planning
- 2026RL-Ballast: Ship Ballast Water Path Planning and Clog Prediction via Reinforcement Learning
- 2025Khalasi: Energy-Efficient Navigation for Surface Vehicles in Vortical Flow Fields