# MPC-based Reinforcement Learning for a Simplified Freight Mission of Autonomous Surface Vehicles

arXiv: https://arxiv.org/abs/2106.08634 · HTML: https://mlsanity.com/paper/2106.08634/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2021.

## Built on by (most cited first, 7 of 7)

- [UAV Path Planning Employing MPC- Reinforcement Learning Method Considering Collision Avoidance](https://mlsanity.com/paper/2302.10669/) (2023)
- [Adaptive Stochastic Nonlinear Model Predictive Control with Look-ahead Deep Reinforcement Learning for Autonomous Vehicle Motion Control](https://mlsanity.com/paper/2311.04303/) (2023)
- [Learning-based MPC from Big Data Using Reinforcement Learning](https://mlsanity.com/paper/2301.01667/) (2023)
- [MPC4RL -- A Software Package for Reinforcement Learning based on Model Predictive Control](https://mlsanity.com/paper/2501.15897/) (2025)
- [Cost-Matching Model Predictive Control for Efficient Reinforcement Learning in Humanoid Locomotion](https://mlsanity.com/paper/2603.28243/) (2026)
- [Mission-Aligned Learning-Informed Control of Autonomous Systems: Formulation and Foundations](https://mlsanity.com/paper/2507.04356/) (2025)
- [Model-free Reinforcement Learning for Model-based Control: Towards Safe, Interpretable and Sample-efficient Agents](https://mlsanity.com/paper/2507.13491/) (2025)

## Abstract (arXiv)

In this work, we propose a Model Predictive Control (MPC)-based Reinforcement Learning (RL) method for Autonomous Surface Vehicles (ASVs). The objective is to find an optimal policy that minimizes the closed-loop performance of a simplified freight mission, including collision-free path following, autonomous docking, and a skillful transition between them. We use a parametrized MPC-scheme to approximate the optimal policy, which considers path-following/docking costs and states (position, velocity)/inputs (thruster force, angle) constraints. The Least Squares Temporal Difference (LSTD)-based Deterministic Policy Gradient (DPG) method is then applied to update the policy parameters. Our simulation results demonstrate that the proposed MPC-LSTD-based DPG method could improve the closed-loop performance during learning for the freight mission problem of ASV.