mlsanityData to 7 Oct 2026

A Policy Optimization Method Towards Optimal-time Stability

arXiv 2301.00521 ↗First posted 2 Jan 2023cs.RO cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In current model-free reinforcement learning (RL) algorithms, stability criteria based on sampling methods are commonly utilized to guide policy optimization. However, these criteria only guarantee the infinite-time convergence of the system's state to an equilibrium point, which leads to sub-optimality of the policy. In this paper, we propose a policy optimization technique incorporating sampling-based Lyapunov stability. Our approach enables the system's state to reach an equilibrium point within an optimal time and maintain stability thereafter, referred to as "optimal-time stability". To achieve this, we integrate the optimization method into the Actor-Critic framework, resulting in the development of the Adaptive Lyapunov-based Actor-Critic (ALAC) algorithm. Through evaluations conducted on ten robotic tasks, our approach outperforms previous studies significantly, effectively guiding the system to generate stable patterns.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 2 Jan 2023.

Builds on

32 of 43 references.

References · 32

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2016OpenAI Gymcited by 1,684
  5. 2017Constrained Policy Optimizationcited by 928
  6. 2018Learning Dexterous In-Hand Manipulationcited by 643
  7. 2019Learning agile and dynamic motor skills for legged robotscited by 402
  8. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  9. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  10. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
  11. 2020Neural Lyapunov Controlcited by 142
  12. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 140
20 more
  1. 2021Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functionscited by 91
  2. 2018The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systemscited by 87
  3. 2019Finite-Sample Analysis for SARSA with Linear Function Approximationcited by 85
  4. 2021Lyapunov-stable neural-network controlcited by 66
  5. 2020Enforcing robust control guarantees within neural network policiescited by 40
  6. 2020Actor-Critic Reinforcement Learning for Control with Stability Guaranteecited by 39
  7. 2022Neural Lyapunov Control of Unknown Nonlinear Systems with Stability Guaranteescited by 39
  8. 2020Neural Certificates for Safe Control Policiescited by 35
  9. 2021Stabilizing Neural Control Using Self-Learned Almost Lyapunov Criticscited by 33
  10. 2021Stability Verification in Stochastic Control Systems via Neural Network Supermartingalescited by 24
  11. 2021Learning Stable Deep Dynamics Models for Partially Observed or Delayed Dynamical Systemscited by 17
  12. 2021Almost Surely Stable Deep Dynamicscited by 16
  13. 2021Lyapunov-Net: A Deep Neural Network Architecture for Lyapunov Function Approximationcited by 16
  14. 2021Learning Region of Attraction for Nonlinear Systemscited by 16
  15. 2021Separated Proportional-Integral Lagrangian for Chance Constrained Reinforcement Learningcited by 12
  16. 2021Lyapunov Barrier Policy Optimizationcited by 12
  17. 2020Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guaranteecited by 7
  18. 2022Model-free Neural Lyapunov Control for Safe Robot Navigationcited by 5
  19. 2022KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systemscited by 5
  20. 2022Learning Stabilizable Deep Dynamics Modelscited by 2

Built on by

Not cited yet.