mlsanityData to 7 Oct 2026

Stability-Constrained Markov Decision Processes Using MPC

arXiv 2102.01383 ↗First posted 2 Feb 2021cs.LG cs.SY eess.SYcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we consider solving discounted Markov Decision Processes (MDPs) under the constraint that the resulting policy is stabilizing. In practice MDPs are solved based on some form of policy approximation. We will leverage recent results proposing to use Model Predictive Control (MPC) as a structured policy in the context of Reinforcement Learning to make it possible to introduce stability requirements directly inside the MPC-based policy. This will restrict the solution of the MDP to stabilizing policies by construction. The stability theory for MPC is most mature for the undiscounted MPC case. Hence, we will first show in this paper that stable discounted MDPs can be reformulated as undiscounted ones. This observation will entail that the MPC-based policy with stability requirements will produce the optimal policy for the discounted MDP if it is stable, and the best stabilizing policy otherwise.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 10 of its 34 references that are in the map. First posted on arXiv 2 Feb 2021.

Builds on

10 of 34 references.

References · 10

  1. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  2. 2019Safe Reinforcement Learning Using Robust MPCcited by 72
  3. 2019Data-driven Economic NMPC using Reinforcement Learningcited by 55
  4. 2018Safely Learning to Control the Constrained Linear Quadratic Regulatorcited by 31
  5. 2020Safe Reinforcement Learning via Projection on a Safe Set: How to Achieve Optimality?cited by 29
  6. 2019Probabilistic model predictive safety certification for learning-based controlcited by 20
  7. 2019Learning-based Model Predictive Control for Safe Exploration and Reinforcement Learningcited by 16
  8. 2019Practical Reinforcement Learning of Stabilizing Economic MPCcited by 15
  9. 2020Reinforcement Learning for Mixed-Integer Problems Based on MPCcited by 11
  10. 2020Reinforcement Learning Based on Real-Time Iteration NMPCcited by 3

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 1.

4 papers.

  1. 2025Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classificationcited by 19
  2. 2024AC4MPC: Actor-Critic Reinforcement Learning for Nonlinear Model Predictive Controlcited by 4
  3. 2023Modeling Risk in Reinforcement Learning: A Literature Mappingcited by 1
  4. 2024Incremental Gauss-Newton Descent for Machine Learningcited by 1