Does Latent Context Help? A Controlled Evaluation of Inverse Reinforcement Learning in Arctic Shipping
On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF
also in 2 more
Abstract (arXiv)
Artificial Intelligence (AI)-assisted navigation can help Arctic shipping adapt to rapidly changing sea-ice conditions, but reliable deployment requires reward models that are interpretable and robust to changing environments. Inverse reinforcement learning (IRL) provides a framework for recovering such rewards from vessel trajectories, while recent meta-IRL methods introduce latent context variables to capture behavioral heterogeneity. However, it remains unclear whether these latent representations recover genuinely hidden preferences or simply re-encode information already available in the observed state. We conduct a controlled evaluation on 3,186 AIS-derived voyages from 202 vessels across nine Arctic shipping seasons, comparing a linear shared reward, a nonlinear shared reward, and a latent-context model built on the same nonlinear architecture. The nonlinear reward improves held-out likelihood by 50.9% over the linear baseline, whereas adding vessel-specific latent context reduces performance by 16.5%. Behavioral analysis, context probes, and a pre-registered feature-hiding ablation show that apparent vessel-level variation is largely explained by observable route and environmental conditions rather than hidden vessel-specific factors. Moreover, predictive accuracy, route fidelity, and reward transfer yield different model rankings, demonstrating that no single metric is sufficient to evaluate learned rewards. These findings motivate testing whether the observed route, environmental, and vessel features already explain behavioral variation before adding per-vessel latent context. This supports more trustworthy AI deployment in safety-critical domains.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 15 of its 37 references that are in the map. First posted on arXiv 6 Aug 2026.
Builds on
15 of 37 references.
References · 15
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
- 2016Concrete Problems in AI Safetycited by 1,642
- 2016Generative Adversarial Imitation Learningcited by 1,597
- 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
- 2018An Algorithmic Perspective on Imitation Learningcited by 459
- 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
- 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
- 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 171
- 2021What Matters for Adversarial Imitation Learning?cited by 70
- 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variablescited by 50
- 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learningcited by 36
3 more
- 2019Empowering A* Search Algorithms with Neural Networks for Personalized Route Recommendationcited by 21
- 2021TrAISformer -- A Transformer Network with Sparse Augmented Data Representation and Cross Entropy Loss for AIS-based Vessel Trajectory Predictioncited by 20
- 2025Goal-Conditioned Reinforcement Learning for Data-Driven Maritime Navigationcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Task-specific Subnetwork Discovery in Reinforcement Learning for Autonomous Underwater Navigation
- 2026Physics-informed offline reinforcement learning eliminates catastrophic fuel waste in maritime routing
- 2026In-Context Reinforcement Learning under Non-Stationarity: A Survey
- 2024In-context Exploration-Exploitation for Reinforcement Learning
- 2020Learning Navigation Costs from Demonstration in Partially Observable Environments
- 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models
- 2026Realistic Curriculum Reinforcement Learning for Autonomous and Sustainable Marine Vessel Navigation
- 2025CREStE: Scalable Mapless Navigation with Internet Scale Priors and Counterfactual Guidance