mlsanityData to 14 Oct 2026

Policy Optimization finds Nash Equilibrium in Regularized General-Sum LQ Games

arXiv 2404.00045 ↗First posted 25 Mar 2024cs.GT cs.AI cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate the impact of introducing relative entropy regularization on the Nash Equilibria (NE) of General-Sum N-agent games, revealing the fact that the NE of such games conform to linear Gaussian policies. Moreover, it delineates sufficient conditions, contingent upon the adequacy of entropy regularization, for the uniqueness of the NE within the game. As Policy Optimization serves as a foundational approach for Reinforcement Learning (RL) techniques aimed at finding the NE, in this work we prove the linear convergence of a policy optimization algorithm which (subject to the adequacy of entropy regularization) is capable of provably attaining the NE. Furthermore, in scenarios where the entropy regularization proves insufficient, we present a δ-augmentation technique, which facilitates the achievement of an ε-NE within the game.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 28 of its 38 references that are in the map. First posted on arXiv 25 Mar 2024.

Builds on

28 of 38 references.

References · 28

  1. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
  2. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 311
  3. 2019Reinforcement Learning in Healthcare: A Surveycited by 310
  4. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  5. 2021V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RLcited by 114
  6. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 73
  7. 2021When Can We Learn General-Sum Markov Games with a Large Number of Players Sample-Efficiently?cited by 67
  8. 2018Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertisingcited by 65
  9. 2020Reinforcement Learning in Economics and Financecited by 58
  10. 2020Entropy Regularization for Mean Field Games with Learningcited by 58
  11. 2020Unified Reinforcement Q-Learning for Mean Field Game and Control Problemscited by 49
  12. 2019Policy Optimization Provably Converges to Nash Equilibria in Zero-Sum Linear Quadratic Gamescited by 42
16 more
  1. 2021Provably Efficient Reinforcement Learning in Decentralized General-Sum Markov Gamescited by 42
  2. 2021Policy Gradient Methods Find the Nash Equilibrium in N-player General-sum Linear-quadratic Gamescited by 25
  3. 2022Oracle-free Reinforcement Learning in Mean-Field Games along a Single Sample Pathcited by 20
  4. 2018Exploration versus exploitation in reinforcement learning: a stochastic control approachcited by 18
  5. 2021Understanding the Effect of Stochasticity in Policy Optimizationcited by 18
  6. 2019Global Convergence of Policy Gradient for Sequential Zero-Sum Linear Quadratic Dynamic Gamescited by 16
  7. 2020Exploratory LQG Mean Field Games with Entropy Regularizationcited by 15
  8. 2023Fast Policy Learning for Linear Quadratic Control with Entropy Regularizationcited by 13
  9. 2023Revisiting LQR Control from the Perspective of Receding-Horizon Policy Gradientcited by 10
  10. 2023Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximationcited by 9
  11. 2024Blending Data-Driven Priors in Dynamic Gamescited by 6
  12. 2022Optimal Correlated Equilibria in General-Sum Extensive-Form Games: Fixed-Parameter Algorithms, Hardness, and Two-Sided Column-Generationcited by 5
  13. 2024Provable Policy Gradient Methods for Average-Reward Markov Potential Gamescited by 5
  14. 2024Independent RL for Cooperative-Competitive Agents: A Mean-Field Perspectivecited by 3
  15. 2020Reinforcement Learning in Nonzero-sum Linear Quadratic Deep Structured Games: Global Convergence of Policy Optimizationcited by 2
  16. 2022Linear Quadratic Mean-Field Games with Communication Constraintscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approachcited by 5
  2. 2025Online Multi-Agent Control with Adversarial Disturbancescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Linear Convergence of Independent Natural Policy Gradient in Games with Entropy Regularization
  2. 2025NashPG: A Policy Gradient Method with Iteratively Refined Regularization for Finding Nash Equilibria
  3. 2026Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Games
  4. 2024A Policy-Gradient Approach to Solving Imperfect-Information Games with Best-Iterate Convergence
  5. 2025Convergence Theorems for Entropy-Regularized and Distributional Reinforcement Learning
  6. 2025Solving Zero-Sum Convex Markov Games
  7. 2026Pessimism-Free Offline Learning in General-Sum Games via KL Regularization
  8. 2026Entropy Regularized Reinforcement Learning for Zero-Sum Stochastic Differential Games in a Regime-Switching Jump-Diffusion Process