mlsanityData to 7 Oct 2026

Zero-Sum Positional Differential Games as a Framework for Robust Reinforcement Learning: Deep Q-Learning Approach

arXiv 2405.02044 ↗First posted 3 May 2024cs.LG cs.AI cs.GTcited by 0

Not on the map. Why?

Abstract (arXiv)

Robust Reinforcement Learning (RRL) is a promising Reinforcement Learning (RL) paradigm aimed at training robust to uncertainty or disturbances models, making them more efficient for real-world applications. Following this paradigm, uncertainty or disturbances are interpreted as actions of a second adversarial agent, and thus, the problem is reduced to seeking the agents' policies robust to any opponent's actions. This paper is the first to propose considering the RRL problems within the positional differential game theory, which helps us to obtain theoretically justified intuition to develop a centralized Q-learning approach. Namely, we prove that under Isaacs's condition (sufficiently general for real-world dynamical systems), the same Q-function can be utilized as an approximate solution of both minimax and maximin Bellman equations. Based on these results, we present the Isaacs Deep Q-Network algorithms and demonstrate their superiority compared to other baseline RRL and Multi-Agent RL algorithms in various environments.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 61 references that are in the map. First posted on arXiv 3 May 2024.

Builds on

21 of 61 references.

References · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  5. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  6. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  7. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  8. 2017Robust Adversarial Reinforcement Learningcited by 439
  9. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 265
  10. 2019A Theoretical Analysis of Deep Q-Learningcited by 212
  11. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  12. 2015Multiagent Cooperation and Competition with Deep Reinforcement Learningcited by 180
9 more
  1. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
  2. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  3. 2019Wasserstein Robust Reinforcement Learningcited by 50
  4. 2020Robust Reinforcement Learning via Adversarial training with Langevin Dynamicscited by 45
  5. 2019The Differentiable Cross-Entropy Methodcited by 26
  6. 2019H_∞ Model-free Reinforcement Learning with Robust Stability Guaranteecited by 7
  7. 2022Infinite-Horizon Reach-Avoid Zero-Sum Games via Deep Reinforcement Learningcited by 6
  8. 2021Reinforcement Learning In Two Player Zero Sum Simultaneous Action Gamescited by 4
  9. 2022A Deep Reinforcement Learning Approach for Finding Non-Exploitable Strategies in Two-Player Atari Gamescited by 3

Built on by

Not cited yet.