mlsanityData to 14 Oct 2026

Safe Reinforcement Learning with Chance-constrained Model Predictive Control

arXiv 2112.13941 ↗First posted 27 Dec 2021cs.LG cs.SY eess.SYcited by 20

Not on the map. Why?

Abstract (arXiv)

Real-world reinforcement learning (RL) problems often demand that agents behave safely by obeying a set of designed constraints. We address the challenge of safe RL by coupling a safety guide based on model predictive control (MPC) with a modified policy gradient framework in a linear setting with continuous actions. The guide enforces safe operation of the system by embedding safety requirements as chance constraints in the MPC formulation. The policy gradient training step then includes a safety penalty which trains the base policy to behave safely. We show theoretically that this penalty allows for a provably safe optimal base policy and illustrate our method with a simulated linearized quadrotor experiment.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 14 of its 30 references that are in the map. First posted on arXiv 27 Dec 2021.

Builds on

14 of 30 references.

References · 14

  1. 2017Constrained Policy Optimizationcited by 983
  2. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  3. 2018Reward Constrained Policy Optimizationcited by 417
  4. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  5. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  6. 2018Safe Exploration in Continuous Action Spacescited by 247
  7. 2016Sample Efficient Actor-Critic with Experience Replaycited by 188
  8. 2017Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximationcited by 121
  9. 2018Learning-based Model Predictive Control for Safe Explorationcited by 120
  10. 2018A predictive safety filter for learning-based control of constrained nonlinear dynamical systemscited by 118
  11. 2018EnsembleDAgger: A Bayesian Approach to Safe Imitation Learningcited by 100
  12. 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 47
2 more
  1. 2018Linear model predictive safety certification for learning-based controlcited by 37
  2. 2013Convex computation of the maximum controlled invariant set for polynomial control systemscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 10; 2025: 2; 2026: 2.

20 papers.

Sort:

By area:

  1. 2024A Survey of Constraint Formulations in Safe Reinforcement Learningcited by 44
  2. 2023Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithmscited by 28
  3. 2025Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classificationcited by 26
  4. 2024Safety Filtering While Training: Improving the Performance and Sample Efficiency of Reinforcement Learning Agentscited by 17
  5. 2024Safe Reinforcement Learning on the Constraint Manifold: Theory and Applicationscited by 15
  6. 2022Targeted Adversarial Attacks against Neural Network Trajectory Predictorscited by 11
  7. 2024A Review of Safe Reinforcement Learning Methods for Modern Power Systemscited by 11
  8. 2025A Probabilistic Neuro-symbolic Layer for Algebraic Constraint Satisfactioncited by 11
  9. 2024Conformal Predictive Programming for Chance Constrained Optimizationcited by 6
  10. 2023A Generalizable Physics-informed Learning Framework for Risk Probability Estimationcited by 4
  11. 2023Initial State Interventions for Deconfounded Imitation Learningcited by 4
  12. 2024A Retention-Centric Framework for Continual Learning with Guaranteed Model Developmental Safetycited by 4
8 more
  1. 2023Tempo Adaptation in Non-stationary Reinforcement Learningcited by 3
  2. 2024Handling Long-Term Safety and Uncertainty in Safe Reinforcement Learningcited by 3
  3. 2024Flipping-based Policy for Chance-Constrained Markov Decision Processescited by 3
  4. 2024DeepSafeMPC: Deep Learning-Based Model Predictive Control for Safe Multi-Agent Reinforcement Learningcited by 2
  5. 2024Generalizable Physics-Informed Learning for Stochastic Safety-Critical Systemscited by 2
  6. 2022Finding Safe Zones of policies Markov Decision Processescited by 1
  7. 2026Bridging Reinforcement Learning and Optimal Control via Feasible Action Mappingcited by 0
  8. 2026Learning Chance-Constrained MDPs with Bellman Distributional Certificatescited by 0