Data to 7 Oct 2026

Accelerated Primal-Dual Policy Optimization for Safe Reinforcement Learning

arXiv 1802.06480 ↗First posted 19 Feb 2018cs.AI cs.LG stat.MLcited by 61

Not on the map. Why?

Abstract (arXiv)

Constrained Markov Decision Process (CMDP) is a natural framework for reinforcement learning tasks with safety constraints, where agents learn a policy that maximizes the long-term reward while satisfying the constraints on the long-term cost. A canonical approach for solving CMDPs is the primal-dual method which updates parameters in primal and dual spaces in turn. Existing methods for CMDPs only use on-policy data for dual updates, which results in sample inefficiency and slow convergence. In this paper, we propose a policy search method for CMDPs called Accelerated Primal-Dual Optimization (APDO), which incorporates an off-policy trained dual variable in the dual update procedure while updating the policy in primal space with on-policy likelihood ratio gradient. Experimental results on a simulated robot locomotion task show that APDO achieves better sample efficiency and faster convergence than state-of-the-art approaches for CMDPs.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 14; 2023: 15; 2024: 15; 2025: 7; 2026: 3.

61 papers.

Sort:

By area:

2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learningcited by 90
2021Learning Policies with Zero or Bounded Constraint Violation for Constrained MDPscited by 78
2022Constrained Variational Policy Optimization for Safe Reinforcement Learningcited by 69
2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 55
2024Agile But Safe: Learning Collision-Free High-Speed Legged Locomotioncited by 53
2023State-wise Safe Reinforcement Learning: A Surveycited by 48
2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
2022Learning Infinite-Horizon Average-Reward Markov Decision Processes with Constraintscited by 29
2022Convergence and sample complexity of natural policy gradient primal-dual methods for constrained MDPscited by 26

38 more

2021Policy Optimization for Constrained MDPs with Provable Fast Global Convergencecited by 22
2023Constraint-Conditioned Policy Optimization for Versatile Safe Reinforcement Learningcited by 17
2022CUP: A Conservative Update Policy Algorithm for Safe Reinforcement Learningcited by 16
2023AutoCost: Evolving Intrinsic Cost for Zero-violation Reinforcement Learningcited by 16
2023Learning Shared Safety Constraints from Multi-task Demonstrationscited by 15
2021Finite-Time Complexity of Online Primal-Dual Natural Actor-Critic Algorithm for Constrained Markov Decision Processescited by 14
2022Robust Constrained Reinforcement Learningcited by 14
2023Probabilistic Constraint for Safety-Critical Reinforcement Learningcited by 11
2022Accuracy-Guaranteed Collaborative DNN Inference in Industrial IoT via Deep Reinforcement Learningcited by 10
2023A safe exploration approach to constrained Markov decision processescited by 10
2026Machine Unlearning under Retain-Forget Entanglementcited by 10
2023State-wise Constrained Policy Optimizationcited by 9
2023Safe Deep Policy Adaptationcited by 9
2022Towards Painless Policy Optimization for Constrained MDPscited by 8
2021Attention Actor-Critic algorithm for Multi-Agent Constrained Co-operative Reinforcement Learningcited by 7
2022Sampling, Communication, and Prediction Co-Design for Synchronizing the Real-World Device and Digital Model in Metaversecited by 7
2023Gradient Shaping for Multi-Constraint Safe Reinforcement Learningcited by 7
2021Towards Disturbance-Free Visual Mobile Manipulationcited by 6
2024Constrained Reinforcement Learning Under Model Mismatchcited by 6
2025Efficient Policy Optimization in Robust Constrained MDPs with Iteration Complexity Guaranteescited by 6
2026Failure-Aware RL: Reliable Offline-to-Online Reinforcement Learning with Self-Recovery for Real-World Manipulationcited by 6
2022Mean-Field Approximation of Cooperative Constrained Multi-Agent Reinforcement Learning (CMARL)cited by 5
2023Prioritized Soft Q-Decomposition for Lexicographic Reinforcement Learningcited by 5
2023Reduced Policy Optimization for Continuous Control with Hard Constraintscited by 5
2022Policy Gradients for Probabilistic Constrained Reinforcement Learningcited by 4
2024Reinforcement Learning with Ensemble Model Predictive Safety Certificationcited by 4
2025Rectified Robust Policy Optimization for Model-Uncertain Constrained Reinforcement Learning without Strong Dualitycited by 4
2023Marketing Budget Allocation with Offline Constrained Deep Reinforcement Learningcited by 3
2024A Safe Reinforcement Learning Algorithm for Supervisory Control of Power Plantscited by 3
2024Event-Triggered Reinforcement Learning Based Joint Resource Allocation for Ultra-Reliable Low-Latency V2X Communicationscited by 3
2023Secure Deep Reinforcement Learning for Dynamic Resource Allocation in Wireless MEC Networkscited by 2
2024Federated Reinforcement Learning with Constraint Heterogeneitycited by 2
2024Safety Constrained Multi-Agent Reinforcement Learning for Active Voltage Controlcited by 2
2024Tilted Quantile Gradient Updates for Quantile-Constrained Reinforcement Learningcited by 2
2025PNAct: Crafting Backdoor Attacks in Safe Reinforcement Learningcited by 2
2025Safe In-Context Reinforcement Learningcited by 2
2023Handling Long and Richly Constrained Tasks through Constrained Hierarchical Reinforcement Learningcited by 1
2024Constrained Optimal Fuel Consumption of HEV: A Constrained Reinforcement Learning Approachcited by 1

50 most cited of 61 All 61 (JSON)