# A General Framework for Learning Mean-Field Games

arXiv: https://arxiv.org/abs/2003.06069 · HTML: https://mlsanity.com/paper/2003.06069/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Mar 2020.

## Built on by (most cited first, 25 of 25)

- [Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning](https://mlsanity.com/paper/2107.04050/) (2021)
- [Scalable Deep Reinforcement Learning Algorithms for Mean Field Games](https://mlsanity.com/paper/2203.11973/) (2022)
- [Generalization in Mean Field Games by Learning Master Policies](https://mlsanity.com/paper/2109.09717/) (2021)
- [Policy Mirror Ascent for Efficient and Independent Learning in Mean Field Games](https://mlsanity.com/paper/2212.14449/) (2022)
- [A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.03859/) (2022)
- [Learning Sparse Graphon Mean Field Games](https://mlsanity.com/paper/2209.03880/) (2022)
- [When is Mean-Field Reinforcement Learning Tractable and Relevant?](https://mlsanity.com/paper/2402.05757/) (2024)
- [Learning Correlated Equilibria in Mean-Field Games](https://mlsanity.com/paper/2208.10138/) (2022)
- [MF-OML: Online Mean-Field Reinforcement Learning with Occupation Measures for Large Population Games](https://mlsanity.com/paper/2405.00282/) (2024)
- [Finding Regularized Competitive Equilibria of Heterogeneous Agent Macroeconomic Models with Reinforcement Learning](https://mlsanity.com/paper/2303.04833/) (2023)
- [Major-Minor Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2303.10665/) (2023)
- [Model-free policy gradient for discrete-time mean-field control](https://mlsanity.com/paper/2601.11217/) (2026)
- [Hypergraphon Mean Field Games](https://mlsanity.com/paper/2203.16223/) (2022)
- [High-dimensional Mean-Field Games by Particle-based Flow Matching](https://mlsanity.com/paper/2512.01172/) (2025)
- [Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learning](https://mlsanity.com/paper/2203.12048/) (2022)
- [Exploiting Approximate Symmetry for Efficient Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2408.15173/) (2024)
- [Learning Mean Field Control on Sparse Graphs](https://mlsanity.com/paper/2501.17079/) (2025)
- [Finite-Sample Convergence Bounds for Trust Region Policy Optimization in Mean-Field Games](https://mlsanity.com/paper/2505.22781/) (2025)
- [Solving Continuous Mean Field Games: Deep Reinforcement Learning for Non-Stationary Dynamics](https://mlsanity.com/paper/2510.22158/) (2025)
- [Reinforcement Learning for SBM Graphon Games with Re-Sampling](https://mlsanity.com/paper/2310.16326/) (2023)
- [Learning in Stackelberg Mean Field Games: A Non-Asymptotic Analysis](https://mlsanity.com/paper/2509.15392/) (2025)
- [Networked Communication for Decentralised Agents in Mean-Field Games](https://mlsanity.com/paper/2306.02766/) (2023)
- [Networked Communication for Mean-Field Games with Function Approximation and Empirical Mean-Field Estimation](https://mlsanity.com/paper/2408.11607/) (2024)
- [Recurrent Structural Policy Gradient for Partially Observable Mean Field Games](https://mlsanity.com/paper/2602.20141/) (2026)
- [Towards Model-Free Learning in Dynamic Population Games: An Application to Karma Economies](https://mlsanity.com/paper/2605.11042/) (2026)

## Abstract (arXiv)

This paper presents a general mean-field game (GMFG) framework for simultaneous learning and decision-making in stochastic games with a large population. It first establishes the existence of a unique Nash Equilibrium to this GMFG, and demonstrates that naively combining reinforcement learning with the fixed-point approach in classical MFGs yields unstable algorithms. It then proposes value-based and policy-based reinforcement learning algorithms (GMF-V and GMF-P, respectively) with smoothed policies, with analysis of their convergence properties and computational complexities. Experiments on an equilibrium product pricing problem demonstrate that GMF-V-Q and GMF-P-TRPO, two specific instantiations of GMF-V and GMF-P, respectively, with Q-learning and TRPO, are both efficient and robust in the GMFG setting. Moreover, their performance is superior in convergence speed, accuracy, and stability when compared with existing algorithms for multi-agent reinforcement learning in the N-player setting.