mlsanityData to 7 Oct 2026

Bounded Robustness in Reinforcement Learning via Lexicographic Objectives

arXiv 2209.15320 ↗First posted 30 Sep 2022cs.LG cs.AI cs.ROcited by 0

Not on the map. Why?

Abstract (arXiv)

Policy robustness in Reinforcement Learning may not be desirable at any cost: the alterations caused by robustness requirements from otherwise optimal policies should be explainable, quantifiable and formally verifiable. In this work we study how policies can be maximally robust to arbitrary observational noise by analysing how they are altered by this noise through a stochastic linear operator interpretation of the disturbances, and establish connections between robustness and properties of the noise kernel and of the underlying MDPs. Then, we construct sufficient conditions for policy robustness, and propose a robustness-inducing scheme, applicable to any policy gradient algorithm, that formally trades off expected policy utility for robustness through lexicographic optimisation, while preserving convergence and sub-optimality in the policy synthesis.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 46 references that are in the map. First posted on arXiv 30 Sep 2022.

Builds on

18 of 46 references.

References · 18

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Adversarial Attacks on Neural Network Policiescited by 314
  3. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  4. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  5. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
  6. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 163
  7. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 152
  8. 2019Global Optimality Guarantees For Policy Gradient Methodscited by 146
  9. 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 144
  10. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
  11. 2017Delving into adversarial attacks on deep policiescited by 104
  12. 2017Vulnerability of Deep Reinforcement Learning to Policy Induction Attackscited by 91
6 more
  1. 2020Plannable Approximations to MDP Homomorphisms: Equivariance under Actionscited by 61
  2. 2019Risk Averse Robust Adversarial Reinforcement Learningcited by 54
  3. 2019Wasserstein Robust Reinforcement Learningcited by 50
  4. 2022Invariance in Policy Optimisation and Partial Identifiability in Reward Learningcited by 46
  5. 2020Robustifying Reinforcement Learning Agents via Action Space Adversarial Trainingcited by 30
  6. 2019Robust Temporal Difference Learning for Critical Domainscited by 7

Built on by

Not cited yet.