# Graphon Mean Field Games with a Representative Player: Analysis and Learning Algorithm

arXiv: https://arxiv.org/abs/2405.08005 · HTML: https://mlsanity.com/paper/2405.08005/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 52 references that are in the map.
- First posted on arXiv 8 May 2024.

## Builds on (most cited first, 23 of 23)

- [Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1802.05438/) (2018)
- [On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning](https://mlsanity.com/paper/1704.00805/) (2017)
- [Finite-Sample Analysis for SARSA with Linear Function Approximation](https://mlsanity.com/paper/1902.02234/) (2019)
- [Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learning](https://mlsanity.com/paper/2102.01585/) (2021)
- [Fictitious Play for Mean Field Games: Continuous Time Analysis and Applications](https://mlsanity.com/paper/2007.03458/) (2020)
- [Unified Reinforcement Q-Learning for Mean Field Game and Control Problems](https://mlsanity.com/paper/2006.13912/) (2020)
- [On the Convergence of Model Free Learning in Mean Field Games](https://mlsanity.com/paper/1907.02633/) (2019)
- [Learning in Mean Field Games: the Fictitious Play](https://mlsanity.com/paper/1507.06280/) (2015)
- [Scalable Deep Reinforcement Learning Algorithms for Mean Field Games](https://mlsanity.com/paper/2203.11973/) (2022)
- [Mean Field Games Flock! The Reinforcement Learning Way](https://mlsanity.com/paper/2105.07933/) (2021)
- [Generalization in Mean Field Games by Learning Master Policies](https://mlsanity.com/paper/2109.09717/) (2021)
- [Learning Graphon Mean Field Games and Approximate Nash Equilibria](https://mlsanity.com/paper/2112.01280/) (2021)
- [Breaking the Curse of Many Agents: Provable Mean Embedding Q-Iteration for Mean-Field Reinforcement Learning](https://mlsanity.com/paper/2006.11917/) (2020)
- [Learning Sparse Graphon Mean Field Games](https://mlsanity.com/paper/2209.03880/) (2022)
- [Stochastic Graphon Games: I. The Static Case](https://mlsanity.com/paper/1911.10664/) (2019)
- [Graphon Mean Field Games and the GMFG Equations](https://mlsanity.com/paper/2008.10216/) (2020)
- [A Game-Theoretic Framework for Autonomous Vehicles Velocity Control: Bridging Microscopic Differential Games and Macroscopic Mean Field Games](https://mlsanity.com/paper/1903.06053/) (2019)
- [Learning Regularized Monotone Graphon Mean-Field Games](https://mlsanity.com/paper/2310.08089/) (2023)
- [Learning Deep Mean Field Games for Modeling Large Population Behavior](https://mlsanity.com/paper/1711.03156/) (2017)
- [Linear Quadratic Graphon Field Games](https://mlsanity.com/paper/2006.03964/) (2020)
- [A label-state formulation of stochastic graphon games and approximate equilibria on large networks](https://mlsanity.com/paper/2204.09642/) (2022)
- [Mean field games via controlled martingale problems: Existence of Markovian equilibria](https://mlsanity.com/paper/1404.2642/) (2014)
- [A Single Online Agent Can Efficiently Learn Mean Field Games](https://mlsanity.com/paper/2405.03718/) (2024)

## Built on by (most cited first, 5 of 5)

- [Stochastic Semi-Gradient Descent for Learning Mean Field Games with Population-Aware Function Approximation](https://mlsanity.com/paper/2408.08192/) (2024)
- [Learning Mean Field Control on Sparse Graphs](https://mlsanity.com/paper/2501.17079/) (2025)
- [Policy Optimization for Continuous-time Linear-Quadratic Graphon Mean Field Games](https://mlsanity.com/paper/2506.05894/) (2025)
- [Mean-Field Reinforcement Learning without Synchrony](https://mlsanity.com/paper/2602.18026/) (2026)
- [All in One: Generative Modeling as Mean-Field Game Design](https://mlsanity.com/paper/2607.23026/) (2026)

## Abstract (arXiv)

We propose a discrete time graphon game formulation on continuous state and action spaces using a representative player to study stochastic games with heterogeneous interaction among agents. This formulation admits both philosophical and mathematical advantages, compared to a widely adopted formulation using a continuum of players. We prove the existence and uniqueness of the graphon equilibrium with mild assumptions, and show that this equilibrium can be used to construct an approximate solution for finite player game on networks, which is challenging to analyze and solve due to curse of dimensionality. An online oracle-free learning algorithm is developed to solve the equilibrium numerically, and sample complexity analysis is provided for its convergence.