mlsanityData to 14 Oct 2026

Policy Iteration for Two-Player General-Sum Stochastic Stackelberg Games

arXiv 2405.06689 ↗First posted 7 May 2024cs.GT cs.LG cs.MAcited by 0

Not on the map. Why?

Abstract (arXiv)

We address two-player general-sum stochastic Stackelberg games (SSGs), where the leader's policy is optimized considering the best-response follower whose policy is optimal for its reward under the leader. Existing policy gradient and value iteration approaches for SSGs do not guarantee monotone improvement in the leader's policy under the best-response follower. Consequently, their performance is not guaranteed when their limits are not stationary Stackelberg equilibria (SSEs), which do not necessarily exist. In this paper, we derive a policy improvement theorem for SSGs under the best-response follower and propose a novel policy iteration algorithm that guarantees monotone improvement in the leader's performance. Additionally, we introduce Pareto-optimality as an extended optimality of the SSE and prove that our method converges to the Pareto front when the leader is myopic.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 22 references that are in the map. First posted on arXiv 7 May 2024.

Builds on

8 of 22 references.

References · 8

  1. 2017Vulnerability of Deep Reinforcement Learning to Policy Induction Attackscited by 91
  2. 2020Adaptive Reward-Poisoning Attacks against Reinforcement Learningcited by 86
  3. 2020Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learningcited by 81
  4. 2019Deceptive Reinforcement Learning Under Adversarial Manipulations on Cost Signalscited by 50
  5. 2019Bi-level Actor-Critic for Multi-agent Coordinationcited by 39
  6. 2020Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamicscited by 38
  7. 2023Online Learning in Stackelberg Games with an Omniscient Followercited by 15
  8. 2021Decentralized Cooperative Reinforcement Learning with Hierarchical Information Structurecited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning in Stackelberg Mean Field Games: A Non-Asymptotic Analysis
  2. 2026Finite-Time Analysis of Q-Value Iteration for General-Sum Stackelberg Games
  3. 2024Convex-Concave Zero-sum Markov Stackelberg Games
  4. 2024A Policy Iteration Algorithm for N-player General-Sum Linear Quadratic Dynamic Games
  5. 2024A Policy-Gradient Approach to Solving Imperfect-Information Games with Best-Iterate Convergence
  6. 2023Stackelberg Batch Policy Learning
  7. 2024ReLExS: Reinforcement Learning Explanations for Stackelberg No-Regret Learners
  8. 2026Actor-Dual-Critic Dynamics for Zero-sum and Identical-Interest Stochastic Games