# Function Approximation for Solving Stackelberg Equilibrium in Large Perfect Information Games

arXiv: https://arxiv.org/abs/2212.14431 · HTML: https://mlsanity.com/paper/2212.14431/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 41 references that are in the map.
- First posted on arXiv 29 Dec 2022.

## Builds on (most cited first, 14 of 14)

- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning](https://mlsanity.com/paper/1711.00832/) (2017)
- [DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker](https://mlsanity.com/paper/1701.01724/) (2017)
- [Learning with Opponent-Learning Awareness](https://mlsanity.com/paper/1709.04326/) (2017)
- [Multi-agent Reinforcement Learning in Sequential Social Dilemmas](https://mlsanity.com/paper/1702.03037/) (2017)
- [Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning](https://mlsanity.com/paper/2206.15378/) (2022)
- [Human-Level Performance in No-Press Diplomacy via Equilibrium Search](https://mlsanity.com/paper/2010.02923/) (2020)
- [No-Press Diplomacy from Scratch](https://mlsanity.com/paper/2110.02924/) (2021)
- [Can Reinforcement Learning Find Stackelberg-Nash Equilibria in General-Sum Markov Games with Myopic Followers?](https://mlsanity.com/paper/2112.13521/) (2021)
- [Learning Nash Equilibrium for General-Sum Markov Games from Batch Data](https://mlsanity.com/paper/1606.08718/) (2016)
- [Deep Reinforcement Learning for Green Security Games with Real-Time Information](https://mlsanity.com/paper/1811.02483/) (2018)
- [Computation of Stackelberg Equilibria of Finite Sequential Games](https://mlsanity.com/paper/1507.07677/) (2015)
- [Double-oracle sampling method for Stackelberg Equilibrium approximation in general-sum extensive-form games](https://mlsanity.com/paper/1909.03934/) (2019)

## Built on by (most cited first, 1 of 1)

- [Efficient Morphology-Control Co-Design via Stackelberg Proximal Policy Optimization](https://mlsanity.com/paper/2603.15388/) (2026)

## Abstract (arXiv)

Function approximation (FA) has been a critical component in solving large zero-sum games. Yet, little attention has been given towards FA in solving textitgeneral-sum extensive-form games, despite them being widely regarded as being computationally more challenging than their fully competitive or cooperative counterparts. A key challenge is that for many equilibria in general-sum games, no simple analogue to the state value function used in Markov Decision Processes and zero-sum games exists. In this paper, we propose learning the textitEnforceable Payoff Frontier (EPF) -- a generalization of the state value function for general-sum games. We approximate the optimal textitStackelberg extensive-form correlated equilibrium by representing EPFs with neural networks and training them by using appropriate backup operations and loss functions. This is the first method that applies FA to the Stackelberg setting, allowing us to scale to much larger games while still enjoying performance guarantees based on FA error. Additionally, our proposed method guarantees incentive compatibility and is easy to evaluate without having to depend on self-play or approximate best-response oracles.