mlsanityData to 7 Oct 2026

Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbations

arXiv 2307.12062 ↗First posted 22 Jul 2023cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Deploying reinforcement learning (RL) systems requires robustness to uncertainty and model misspecification, yet prior robust RL methods typically only study noise introduced independently across time. However, practical sources of uncertainty are usually coupled across time. We formally introduce temporally-coupled perturbations, presenting a novel challenge for existing robust RL methods. To tackle this challenge, we propose GRAD, a novel game-theoretic approach that treats the temporally-coupled robust RL problem as a partially observable two-player zero-sum game. By finding an approximate equilibrium within this game, GRAD optimizes for general robustness against temporally-coupled perturbations. Experiments on continuous control tasks demonstrate that, compared with prior methods, our approach achieves a higher degree of robustness to various types of attacks on different attack domains, both in settings with temporally-coupled perturbations and decoupled perturbations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 52 of its 79 references that are in the map. First posted on arXiv 22 Jul 2023.

Builds on

52 of 79 references.

References · 52

  1. 2017Robust Adversarial Reinforcement Learningcited by 439
  2. 2017Adversarial Attacks on Neural Network Policiescited by 314
  3. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 265
  4. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  5. 2017DeepStack: Expert-Level Artificial Intelligence in No-Limit Pokercited by 228
  6. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  7. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
  8. 2016Deep Reinforcement Learning from Self-Play in Imperfect-Information Gamescited by 153
  9. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 152
  10. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
  11. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 129
  12. 2020Combining Deep Reinforcement Learning and Search for Imperfect-Information Gamescited by 110
38 more
  1. 2017Delving into adversarial attacks on deep policiescited by 104
  2. 2018Deep Counterfactual Regret Minimizationcited by 102
  3. 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 76
  4. 2020Robust Deep Reinforcement Learning through Adversarial Losscited by 73
  5. 2022Safe Reinforcement Learning by Imagining the Near Futurecited by 67
  6. 2021DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learningcited by 65
  7. 2020Robust Reinforcement Learning using Adversarial Populationscited by 64
  8. 2022A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Gamescited by 64
  9. 2020From Poincar'e Recurrence to Convergence in Imperfect Information Games: Finding Equilibrium via Regularizationcited by 59
  10. 2020Suphx: Mastering Mahjong with Deep Reinforcement Learningcited by 55
  11. 2021Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RLcited by 54
  12. 2019A Generalized Training Approach for Multiagent Learningcited by 53
  13. 2019Certified Adversarial Robustness for Deep Reinforcement Learningcited by 47
  14. 2020Pipeline PSRO: A Scalable Approach for Finding Approximate Nash Equilibria in Large Gamescited by 47
  15. 2023Improved Sample Complexity Bounds for Distributionally Robust Reinforcement Learningcited by 44
  16. 2019Finding Friend and Foe in Multi-Agent Gamescited by 42
  17. 2022Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learningcited by 42
  18. 2020Human-Level Performance in No-Press Diplomacy via Equilibrium Searchcited by 41
  19. 2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 41
  20. 2022Provable Reinforcement Learning with a Short-Term Memorycited by 40
  21. 2017Safe and Nested Subgame Solving for Imperfect-Information Gamescited by 39
  22. 2021Policy Smoothing for Provably Robust Reinforcement Learningcited by 39
  23. 2020DREAM: Deep Regret minimization with Advantage baselines and Model-free learningcited by 36
  24. 2017Whatever Does Not Kill Deep Reinforcement Learning, Makes It Strongercited by 35
  25. 2018Actor-Critic Policy Optimization in Partially Observable Multiagent Environmentscited by 35
  26. 2023Natural Actor-Critic for Robust Reinforcement Learning with Function Approximationcited by 35
  27. 2021XDO: A Double Oracle Algorithm for Extensive-Form Gamescited by 34
  28. 2019Characterizing Attacks on Deep Reinforcement Learningcited by 31
  29. 2013Solving Imperfect Information Games Using Decompositioncited by 27
  30. 2019Online Robustness Training for Deep Reinforcement Learningcited by 26
  31. 2022Robust Reinforcement Learning as a Stackelberg Game via Adaptively-Regularized Adversarial Trainingcited by 26
  32. 2018Depth-Limited Solving for Imperfect-Information Gamescited by 24
  33. 2022ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regretcited by 24
  34. 2019Single Deep Counterfactual Regret Minimizationcited by 20
  35. 2020The Advantage Regret-Matching Actor-Criticcited by 17
  36. 2021Investigating Vulnerabilities of Deep Neural Policiescited by 15
  37. 2019Curious iLQR: Resolving Uncertainty in Model-based RLcited by 11
  38. 2022Self-Play PSRO: Toward Optimal Populations in Two-Player Zero-Sum Gamescited by 11

50 most cited of 52 All 52 (JSON)

Built on by

Not cited yet.