# Stochastic Potential Games

arXiv: https://arxiv.org/abs/2005.13527 · HTML: https://mlsanity.com/paper/2005.13527/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2020.

## Built on by (most cited first, 4 of 4)

- [Global Convergence of Multi-Agent Policy Gradient in Markov Potential Games](https://mlsanity.com/paper/2106.01969/) (2021)
- [Independent Natural Policy Gradient Always Converges in Markov Potential Games](https://mlsanity.com/paper/2110.10614/) (2021)
- [Gradient play in stochastic games: stationary points, convergence, and sample complexity](https://mlsanity.com/paper/2106.00198/) (2021)
- [Learning Individual Policies in Large Multi-agent Systems through Local Variance Minimization](https://mlsanity.com/paper/2212.13379/) (2022)

## Abstract (arXiv)

Computing the Nash equilibrium (NE) for N-player non-zerosum stochastic games is a formidable challenge. Currently, algorithmic methods in stochastic game theory are unable to compute NE for stochastic games (SGs) for settings in all but extreme cases in which the players either play as a team or have diametrically opposed objectives in a two-player setting. This greatly impedes the application of the SG framework to numerous problems within economics and practical systems of interest. In this paper, we provide a method of computing Nash equilibria in nonzero-sum settings and for populations of players more than two. In particular, we identify a subset of SGs known as stochastic potential games (SPGs) for which the (Markov perfect) Nash equilibrium can be computed tractably and in polynomial time. Unlike SGs for which, in general, computing the NE is PSPACE-hard, we show that SGs with the potential property are P-Complete. We further demonstrate that for each SPG there is a dual Markov decision process whose solution coincides with the MP-NE of the SPG. We lastly provide algorithms that tractably compute the MP-NE for SGs with more than two players.