mlsanityData to 14 Oct 2026

MPC4RL -- A Software Package for Reinforcement Learning based on Model Predictive Control

arXiv 2501.15897 ↗First posted 27 Jan 2025eess.SY cs.SYcited by 2

On the map: Deep reinforcement learning methods → RL for autonomous driving and traffic

Abstract (arXiv)

In this paper, we present an early software integrating Reinforcement Learning (RL) with Model Predictive Control (MPC). Our aim is to make recent theoretical contributions from the literature more accessible to both the RL and MPC communities. We combine standard software tools developed by the RL community, such as Gymnasium, stable-baselines3, or CleanRL with the acados toolbox, a widely-used software package for efficient MPC algorithms. Our core contribution is MPC4RL, an open-source Python package that supports learning-enhanced MPC schemes for existing acados implementations. The package is designed to be modular, extensible, and user-friendly, facilitating the tuning of MPC algorithms for a broad range of control problems. It is available on GitHub.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods. Builds on 11 of its 26 references that are in the map. First posted on arXiv 27 Jan 2025.

Builds on

11 of 26 references.

References · 11

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2016OpenAI Gymcited by 1,736
  4. 2018Differentiable MPC for End-to-end Planning and Controlcited by 189
  5. 2019Safe Reinforcement Learning Using Robust MPCcited by 85
  6. 2019Data-driven Economic NMPC using Reinforcement Learningcited by 68
  7. 2019acados: a modular open-source framework for fast embedded optimal controlcited by 68
  8. 2020HPIPM: a high-performance quadratic programming framework for model predictive controlcited by 23
  9. 2021MPC-based Reinforcement Learning for Economic Problems with Application to Battery Storagecited by 8
  10. 2021MPC-based Reinforcement Learning for a Simplified Freight Mission of Autonomous Surface Vehiclescited by 7
  11. 2022Equivalence of Optimality Criteria for Markov Decision Process and Model Predictive Controlcited by 3

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Differentiable Nonlinear Model Predictive Controlcited by 6
  2. 2025Model-free Reinforcement Learning for Model-based Control: Towards Safe, Interpretable and Sample-efficient Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MPCritic: A plug-and-play MPC architecture for reinforcement learning
  2. 2026A Systematic Review and Taxonomy of Reinforcement Learning-Model Predictive Control Integration for Linear Systems
  3. 2025Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification
  4. 2025Hierarchical Reinforcement Learning with Low-Level MPC for Multi-Agent Control
  5. 2024AC4MPC: Actor-Critic Reinforcement Learning for Nonlinear Model Predictive Control
  6. 2022A Survey on Model-based Reinforcement Learning
  7. 2024Model Predictive Control and Reinforcement Learning: A Unified Framework Based on Dynamic Programming
  8. 2024Towards an Adaptable and Generalizable Optimization Engine in Decision and Control: A Meta Reinforcement Learning Approach