mlsanityData to 7 Oct 2026

Lyapunov Barrier Policy Optimization

arXiv 2103.09230 ↗First posted 16 Mar 2021cs.LG cs.AI cs.ROcited by 12

Not on the map. Why?

Abstract (arXiv)

Deploying Reinforcement Learning (RL) agents in the real-world require that the agents satisfy safety constraints. Current RL agents explore the environment without considering these constraints, which can lead to damage to the hardware or even other agents in the environment. We propose a new method, LBPO, that uses a Lyapunov-based barrier function to restrict the policy update to a safe set for each training iteration. Our method also allows the user to control the conservativeness of the agent with respect to the constraints in the environment. LBPO significantly outperforms state-of-the-art baselines in terms of the number of constraint violations during training while being competitive in terms of performance. Further, our analysis reveals that baselines like CPO and SDDPG rely mostly on backtracking to ensure safety rather than safe projection, which provides insight into why previous methods might not have effectively limit the number of constraint violations.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 18 of its 33 references that are in the map. First posted on arXiv 16 Mar 2021.

Builds on

18 of 33 references.

References · 18

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2017Constrained Policy Optimizationcited by 928
  4. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  5. 2019Solving Rubik's Cube with a Robot Handcited by 699
  6. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  7. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  8. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
  9. 2018Safe Exploration in Continuous Action Spacescited by 237
  10. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 140
  11. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 109
  12. 2014Optimizing the CVaR via Samplingcited by 98
6 more
  1. 2014Algorithms for CVaR Optimization in MDPscited by 98
  2. 2017OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real Worldcited by 64
  3. 2019Being Optimistic to Be Conservative: Quickly Learning a CVaR Policycited by 63
  4. 2019Worst Cases Policy Gradientscited by 52
  5. 2019Safe Exploration for Interactive Machine Learningcited by 51
  6. 2020Cautious Adaptation For Reinforcement Learning in Safety-Critical Settingscited by 42

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 1; 2023: 2; 2024: 3; 2025: 0; 2026: 2.

12 papers.

  1. 2021Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violationscited by 36
  2. 2021Safe Driving via Expert Guided Policy Optimizationcited by 29
  3. 2022Model-based Safe Deep Reinforcement Learning via a Constrained Proximal Policy Optimization Algorithmcited by 27
  4. 2021Safe Exploration by Solving Early Terminated MDPcited by 13
  5. 2024Safe Reinforcement Learning on the Constraint Manifold: Theory and Applicationscited by 12
  6. 2023Efficient Off-Policy Safe Reinforcement Learning Using Trust Region Conditional Value at Riskcited by 11
  7. 2021Improving Safety in Deep Reinforcement Learning using Unsupervised Action Planningcited by 7
  8. 2024Reinforcement Learning with Ensemble Model Predictive Safety Certificationcited by 4
  9. 2024Handling Long-Term Safety and Uncertainty in Safe Reinforcement Learningcited by 2
  10. 2023A Policy Optimization Method Towards Optimal-time Stabilitycited by 0
  11. 2026HALO: Learning Human-Robot Collaboration via Heterogeneous-Agent Lyapunov Policy Optimizationcited by 0
  12. 2026Interaction-Aware Whole-Body Control for Compliant Object Transportcited by 0