mlsanityData to 7 Oct 2026

Efficient Action Robust Reinforcement Learning with Probabilistic Policy Execution Uncertainty

arXiv 2307.07666 ↗First posted 15 Jul 2023cs.LG cs.AI math.OCcited by 2

Not on the map. Why?

Abstract (arXiv)

Robust reinforcement learning (RL) aims to find a policy that optimizes the worst-case performance in the face of uncertainties. In this paper, we focus on action robust RL with the probabilistic policy execution uncertainty, in which, instead of always carrying out the action specified by the policy, the agent will take the action specified by the policy with probability 1-ρ and an alternative adversarial action with probability ρ. We establish the existence of an optimal policy on the action robust MDPs with probabilistic policy execution uncertainty and provide the action robust Bellman optimality equation for its solution. Furthermore, we develop Action Robust Reinforcement Learning with Certificates (ARRLC) algorithm that achieves minimax optimal regret and sample complexity. Furthermore, we conduct numerical experiments to validate our approach's robustness, demonstrating that ARRLC outperforms non-robust RL algorithms and converges faster than the robust TD algorithm in the presence of action perturbations.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 33 of its 45 references that are in the map. First posted on arXiv 15 Jul 2023.

Builds on

33 of 45 references.

References · 33

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2017Minimax Regret Bounds for Reinforcement Learningcited by 500
  4. 2018Is Q-learning Provably Efficient?cited by 485
  5. 2017Robust Adversarial Reinforcement Learningcited by 439
  6. 2018Reinforcement Learning for Solving the Vehicle Routing Problemcited by 327
  7. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  8. 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 175
  9. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
  10. 2020Almost Optimal Model-Free Reinforcement Learning via Reference-Advantage Decompositioncited by 127
  11. 2021V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RLcited by 111
  12. 2019Non-Asymptotic Gap-Dependent Regret Bounds for Tabular MDPscited by 104
21 more
  1. 2020A Sharp Analysis of Model-based Reinforcement Learning with Self-Playcited by 104
  2. 2018Deep Reinforcement Learning for Page-wise Recommendationscited by 99
  3. 2020Is Reinforcement Learning More Difficult Than Bandits? A Near-optimal Algorithm Escaping the Curse of Horizoncited by 99
  4. 2021Online Robust Reinforcement Learning with Model Uncertaintycited by 95
  5. 2020Sample Complexity of Asynchronous Q-Learning: Sharper Analysis and Variance Reductioncited by 93
  6. 2020Adaptive Reward-Poisoning Attacks against Reinforcement Learningcited by 86
  7. 2020FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Financecited by 86
  8. 2022Distributionally Robust Model-Based Offline Reinforcement Learning with Near-Optimal Sample Complexitycited by 85
  9. 2021Sample Complexity of Robust Reinforcement Learning with a Generative Modelcited by 82
  10. 2018Policy Certificates: Towards Accountable Reinforcement Learningcited by 81
  11. 2018Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulationcited by 76
  12. 2021Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RLcited by 54
  13. 2019Deceptive Reinforcement Learning Under Adversarial Manipulations on Cost Signalscited by 50
  14. 2023The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Modelcited by 50
  15. 2023Improved Sample Complexity Bounds for Distributionally Robust Reinforcement Learningcited by 44
  16. 2020Nearly Minimax Optimal Reinforcement Learning for Discounted MDPscited by 42
  17. 2020Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamicscited by 38
  18. 2019Non-Stationary Markov Decision Processes, a Worst-Case Approach using Model-Based Reinforcement Learning, Extended versioncited by 36
  19. 2021Provably Efficient Black-Box Action Poisoning Attacks Against Reinforcement Learningcited by 29
  20. 2022Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Beliefcited by 22
  21. 2019Robust Temporal Difference Learning for Critical Domainscited by 7

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 36
  2. 2025Robust Reinforcement Learning in Finance: Modeling Market Impact with Elliptic Uncertainty Setscited by 0