mlsanityData to 14 Oct 2026

Model-Based Safe Reinforcement Learning with Time-Varying State and Control Constraints: An Application to Intelligent Vehicles

arXiv 2112.11217 ↗First posted 18 Dec 2021cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Recently, safe reinforcement learning (RL) with the actor-critic structure for continuous control tasks has received increasing attention. It is still challenging to learn a near-optimal control policy with safety and convergence guarantees. Also, few works have addressed the safe RL algorithm design under time-varying safety constraints. This paper proposes a safe RL algorithm for optimal control of nonlinear systems with time-varying state and control constraints. In the proposed approach, we construct a novel barrier force-based control policy structure to guarantee control safety. A multi-step policy evaluation mechanism is proposed to predict the policy's safety risk under time-varying safety constraints and guide the policy to update safely. Theoretical results on stability and robustness are proven. Also, the convergence of the actor-critic implementation is analyzed. The performance of the proposed algorithm outperforms several state-of-the-art RL algorithms in the simulated Safety Gym environment. Furthermore, the approach is applied to the integrated path following and collision avoidance problem for two real-world intelligent vehicles. A differential-drive vehicle and an Ackermann-drive one are used to verify offline deployment and online learning performance, respectively. Our approach shows an impressive sim-to-real transfer capability and a satisfactory online control performance in the experiment.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 35 of its 79 references that are in the map. First posted on arXiv 18 Dec 2021.

Builds on

35 of 79 references.

References · 35

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2021Convex optimizationcited by 2,515
  4. 2017Constrained Policy Optimizationcited by 983
  5. 2017Safe Reinforcement Learning via Shieldingcited by 449
  6. 2018Reward Constrained Policy Optimizationcited by 417
  7. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  8. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  9. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  10. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
  11. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 145
  12. 2020Provably Efficient Safe Exploration via Primal-Dual Policy Optimizationcited by 134
23 more
  1. 2020CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guaranteecited by 134
  2. 2020Learning to be Safe: Deep RL with a Safety Criticcited by 116
  3. 2020Learning to Utilize Shaping Rewards: A New Approach of Reward Shapingcited by 114
  4. 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 103
  5. 2018The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systemscited by 88
  6. 2019Safe Reinforcement Learning Using Robust MPCcited by 85
  7. 2019Safe Policies for Reinforcement Learning via Primal-Dual Methodscited by 79
  8. 2020Safe Reinforcement Learning via Curriculum Inductioncited by 74
  9. 2019Convergent Policy Optimization for Safe Reinforcement Learningcited by 57
  10. 2020Safety-Critical Model Predictive Control with Discrete-Time Control Barrier Functioncited by 49
  11. 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 47
  12. 2021Safe Reinforcement Learning with Linear Function Approximationcited by 35
  13. 2021Context-Aware Safe Reinforcement Learning for Non-Stationary Environmentscited by 33
  14. 2021Model-based Constrained Reinforcement Learning using Generalized Control Barrier Functioncited by 32
  15. 2020SAMBA: Safe Model-Based & Active Reinforcement Learningcited by 30
  16. 2020Model Predictive Contouring Control for Collision Avoidance in Unstructured Dynamic Environmentscited by 21
  17. 2016Stochastic Primal-Dual Methods and Sample Complexity of Reinforcement Learningcited by 15
  18. 2021Safe Reinforcement Learning Using Robust Action Governorcited by 14
  19. 2020Safe reinforcement learning for probabilistic reachability and safety specifications: A Lyapunov-based approachcited by 13
  20. 2020Accelerating Safe Reinforcement Learning with Constraint-mismatched Policiescited by 11
  21. 2021Robust Tube-based Model Predictive Control with Koopman Operators--Extended Versioncited by 10
  22. 2020Safe Reinforcement Learning of Control-Affine Systems with Vertex Networkscited by 8
  23. 2019Dynamic Control of a Fiber Manufacturing Process using Deep Reinforcement Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 2.

4 papers.

  1. 2024Toward Scalable Multirobot Control: Fast Policy Learning in Distributed MPCcited by 2
  2. 2026Learning Predictive Control with Deep Koopman Operators for Autonomous Vehicle Motion Planningcited by 1
  3. 2024A Nested Graph Reinforcement Learning-based Decision-making Strategy for Eco-platooningcited by 0
  4. 2026Learning-based control of a single-DOF Aero systemcited by 0