# Reinforcement Learning based on Scenario-tree MPC for ASVs

arXiv: https://arxiv.org/abs/2103.11949 · HTML: https://mlsanity.com/paper/2103.11949/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2021.

## Built on by (most cited first, 8 of 8)

- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [MPC-based Reinforcement Learning for Economic Problems with Application to Battery Storage](https://mlsanity.com/paper/2104.02411/) (2021)
- [Learning Stochastic Parametric Differentiable Predictive Control Policies](https://mlsanity.com/paper/2203.01447/) (2022)
- [Approximate Robust NMPC using Reinforcement Learning](https://mlsanity.com/paper/2104.02743/) (2021)
- [Convergent NMPC-based Reinforcement Learning Using Deep Expected Sarsa and Nonlinear Temporal Difference Learning](https://mlsanity.com/paper/2502.04925/) (2025)
- [Mission-Aligned Learning-Informed Control of Autonomous Systems: Formulation and Foundations](https://mlsanity.com/paper/2507.04356/) (2025)
- [Model-free Reinforcement Learning for Model-based Control: Towards Safe, Interpretable and Sample-efficient Agents](https://mlsanity.com/paper/2507.13491/) (2025)
- [Diffusion-Based Scenario Tree Generation for Multivariate Time Series Prediction and Multistage Stochastic Optimization](https://mlsanity.com/paper/2509.14832/) (2025)

## Abstract (arXiv)

In this paper, we present the use of Reinforcement Learning (RL) based on Robust Model Predictive Control (RMPC) for the control of an Autonomous Surface Vehicle (ASV). The RL-MPC strategy is utilized for obstacle avoidance and target (set-point) tracking. A scenario-tree robust MPC is used to handle potential failures of the ship thrusters. Besides, the wind and ocean current are considered as unknown stochastic disturbances in the real system, which are handled via constraints tightening. The tightening and other cost parameters are adjusted by RL, using a Q-learning technique. An economic cost is considered, minimizing the time and energy required to achieve the ship missions. The method is illustrated in simulation on a nonlinear 3-DOF model of a scaled version of the Cybership 2.