mlsanityData to 7 Oct 2026

Global Convergence of Policy Gradient for Sequential Zero-Sum Linear Quadratic Dynamic Games

arXiv 1911.04672 ↗First posted 12 Nov 2019eess.SY cs.SY math.OCcited by 14

Not on the map. Why?

Abstract (arXiv)

We propose projection-free sequential algorithms for linear-quadratic dynamics games. These policy gradient based algorithms are akin to Stackelberg leadership model and can be extended to model-free settings. We show that if the leader performs natural gradient descent/ascent, then the proposed algorithm has a global sublinear convergence to the Nash equilibrium. Moreover, if the leader adopts a quasi-Newton policy, the algorithm enjoys a global quadratic convergence. Along the way, we examine and clarify the intricacies of adopting sequential policy updates for LQ games, namely, issues pertaining to stabilization, indefinite cost structure, and circumventing projection steps.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 3; 2023: 2; 2024: 2; 2025: 1.

14 papers.

Sort:

  1. 2021Independent Policy Gradient Methods for Competitive Reinforcement Learningcited by 97
  2. 2022Independent Policy Gradient for Large-Scale Markov Potential Games: Sharper Rates, Function Approximation, and Game-Agnostic Convergencecited by 54
  3. 2021The Confluence of Networks, Games and Learningcited by 26
  4. 2021Independent Learning in Stochastic Gamescited by 20
  5. 2021Policy Gradient Methods Find the Nash Equilibrium in N-player General-sum Linear-quadratic Gamescited by 19
  6. 2021Provably Efficient Policy Optimization for Two-Player Zero-Sum Markov Gamescited by 18
  7. 2021Derivative-Free Policy Optimization for Linear Risk-Sensitive and Robust Control Design: Implicit Regularization and Sample Complexitycited by 15
  8. 2022Symmetric (Optimistic) Natural Policy Gradient for Multi-agent Learning with Parameter Convergencecited by 8
  9. 2022Towards a Theoretical Foundation of Policy Optimization for Learning Control Policiescited by 6
  10. 2023Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithmscited by 4
  11. 2023Learning Zero-Sum Linear Quadratic Games with Improved Sample Complexity and Last-Iterate Convergencecited by 3
  12. 2024Independent RL for Cooperative-Competitive Agents: A Mean-Field Perspectivecited by 2
2 more
  1. 2024Policy Optimization finds Nash Equilibrium in Regularized General-Sum LQ Gamescited by 2
  2. 2025Online Multi-Agent Control with Adversarial Disturbancescited by 0