Primal-Only Actor Critic Algorithm for Robust Constrained Average Cost MDPs
Not on the map. Why?
Abstract (arXiv)
In this work, we study the problem of finding robust and safe policies in Robust Constrained Average-Cost Markov Decision Processes (RCMDPs). A key challenge in this setting is the lack of strong duality, which prevents the direct use of standard primal-dual methods for constrained RL. Additional difficulties arise from the average-cost setting, where the Robust Bellman operator is not a contraction under any norm. To address these challenges, we propose an actor-critic algorithm for Average-Cost RCMDPs. We show that our method achieves both (ε)-feasibility and (ε)-optimality, and we establish a sample complexities of (tildeOleft(ε-4right)) and (tildeOleft(ε-6right)) with and without slackness assumption, which is comparable to the discounted setting.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 45 references that are in the map. First posted on arXiv 7 Nov 2025.
Builds on
34 of 45 references.
References · 34
- 2018Safe Exploration in Continuous Action Spacescited by 237
- 2020Projection-Based Constrained Policy Optimizationcited by 195
- 2016Distributionally Robust Stochastic Optimization with Wasserstein Distancecited by 182
- 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 144
- 2020CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guaranteecited by 128
- 2019IPO: Interior-point Policy Optimization under Constraintscited by 119
- 2021Sample Complexity of Robust Reinforcement Learning with a Generative Modelcited by 82
- 2019Safe Policies for Reinforcement Learning via Primal-Dual Methodscited by 75
- 2019DeepPool: Distributed Model-free Algorithm for Ride-sharing using Deep Reinforcement Learningcited by 54
- 2021Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Primal-Dual Approachcited by 46
- 2022Policy Gradient in Robust MDPs with Global Convergence Guaranteecited by 35
- 2023Natural Actor-Critic for Robust Reinforcement Learning with Function Approximationcited by 35
22 more
- 2022First-order Policy Optimization for Robust Markov Decision Processcited by 31
- 2022Learning Infinite-Horizon Average-Reward Markov Decision Processes with Constraintscited by 29
- 2023Sample Complexity of Variance-reduced Distributionally Robust Q-learningcited by 28
- 2020Robust Constrained-MDPs: Soft-Constrained Robust Policy Optimization under Model Uncertaintycited by 23
- 2023Robust Average-Reward Markov Decision Processescited by 22
- 2023Model-Free Robust Average-Reward Reinforcement Learningcited by 20
- 2022Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Conservative Natural Policy Gradient Primal-Dual Algorithmcited by 19
- 2021Concave Utility Reinforcement Learning with Zero-Constraint Violationscited by 18
- 2020Robust Constrained Reinforcement Learning for Continuous Control with Model Misspecificationcited by 16
- 2022Option-Aware Adversarial Inverse Reinforcement Learning for Robotic Controlcited by 16
- 2024Distributionally Robust Constrained Reinforcement Learning under Strong Dualitycited by 15
- 2022Robust Constrained Reinforcement Learningcited by 14
- 2024Sample-Efficient Constrained Reinforcement Learning with General Parameterizationcited by 11
- 2024Near-Optimal Policy Identification in Robust Constrained Markov Decision Processes via Epigraph Formcited by 10
- 2025On The Sample Complexity Bounds In Bilevel Reinforcement Learningcited by 10
- 2024Learning General Parameterized Policies for Infinite Horizon Average Reward Constrained MDPs via Primal-Dual Policy Gradient Algorithmcited by 9
- 2025Global Convergence for Average Reward Constrained MDPs with Primal-Dual Actor Critic Algorithmcited by 8
- 2025Sample Complexity of Distributionally Robust Average-Reward Reinforcement Learningcited by 7
- 2025Finite-Sample Analysis of Policy Evaluation for Robust Average Reward Reinforcement Learningcited by 6
- 2025Efficient Policy Optimization in Robust Constrained MDPs with Iteration Complexity Guaranteescited by 6
- 2025Rectified Robust Policy Optimization for Model-Uncertain Constrained Reinforcement Learning without Strong Dualitycited by 4
- 2025Efficient Q-Learning and Actor-Critic Methods for Robust Average-Reward Reinforcement Learningcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.