# Learning in Mean Field Games: A Survey

arXiv: https://arxiv.org/abs/2205.12944 · HTML: https://mlsanity.com/paper/2205.12944/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2022.

## Built on by (most cited first, 30 of 61)

- [GAM-Agent: Game-Theoretic and Uncertainty-Aware Collaboration for Complex Visual Reasoning](https://mlsanity.com/paper/2505.23399/) (2025)
- [Recent Developments in Machine Learning Methods for Stochastic Control and Games](https://mlsanity.com/paper/2303.10257/) (2023)
- [A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.03859/) (2022)
- [Unified continuous-time q-learning for mean-field game and mean-field control problems](https://mlsanity.com/paper/2407.04521/) (2024)
- [MF-LLM: Simulating Population Decision Dynamics via a Mean-Field Large Language Model Framework](https://mlsanity.com/paper/2504.21582/) (2025)
- [Model-Based RL for Mean-Field Games is not Statistically Harder than Single-Agent RL](https://mlsanity.com/paper/2402.05724/) (2024)
- [Partially Observable Mean Field Multi-Agent Reinforcement Learning Based on Graph-Attention](https://mlsanity.com/paper/2304.12653/) (2023)
- [Efficient Model-Based Concave Utility Reinforcement Learning through Greedy Mirror Descent](https://mlsanity.com/paper/2311.18346/) (2023)
- [Decision-Dependent Stochastic Optimization: The Role of Distribution Dynamics](https://mlsanity.com/paper/2503.07324/) (2025)
- [Population-aware Online Mirror Descent for Mean-Field Games by Deep Reinforcement Learning](https://mlsanity.com/paper/2403.03552/) (2024)
- [Stochastic Semi-Gradient Descent for Learning Mean Field Games with Population-Aware Function Approximation](https://mlsanity.com/paper/2408.08192/) (2024)
- [A brief review of the Deep BSDE method for solving high-dimensional partial differential equations](https://mlsanity.com/paper/2505.17032/) (2025)
- [Major-Minor Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2303.10665/) (2023)
- [Safe Model-Based Multi-Agent Mean-Field Reinforcement Learning](https://mlsanity.com/paper/2306.17052/) (2023)
- [Learning Regularized Graphon Mean-Field Games with Unknown Graphons](https://mlsanity.com/paper/2310.17531/) (2023)
- [Maximum Causal Entropy IRL in Mean-Field Games and GNEP Framework for Forward RL](https://mlsanity.com/paper/2401.06566/) (2024)
- [Population-aware Online Mirror Descent for Mean-Field Games with Common Noise by Deep Reinforcement Learning](https://mlsanity.com/paper/2509.03030/) (2025)
- [Continuous-time q-learning for mean-field control with common noise, part-II: q-learning algorithms](https://mlsanity.com/paper/2604.27378/) (2026)
- [Learning Mean Field Games on Sparse Graphs: A Hybrid Graphex Approach](https://mlsanity.com/paper/2401.12686/) (2024)
- [Independent RL for Cooperative-Competitive Agents: A Mean-Field Perspective](https://mlsanity.com/paper/2403.11345/) (2024)
- [Analysis of Multiscale Reinforcement Q-Learning Algorithms for Mean Field Control Games](https://mlsanity.com/paper/2405.17017/) (2024)
- [Kernel Based Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games](https://mlsanity.com/paper/2507.14529/) (2025)
- [End-to-End Training of High-Dimensional Optimal Control with Implicit Hamiltonians via Jacobian-Free Backpropagation](https://mlsanity.com/paper/2510.00359/) (2025)
- [Convergence of Actor-Critic Learning for Mean Field Games and Mean Field Control in Continuous Spaces](https://mlsanity.com/paper/2511.06812/) (2025)
- [A Single Online Agent Can Efficiently Learn Mean Field Games](https://mlsanity.com/paper/2405.03718/) (2024)
- [Reinforcement Learning for Finite Space Mean-Field Type Games](https://mlsanity.com/paper/2409.18152/) (2024)
- [Learning Mean Field Control on Sparse Graphs](https://mlsanity.com/paper/2501.17079/) (2025)
- [Finite-Sample Convergence Bounds for Trust Region Policy Optimization in Mean-Field Games](https://mlsanity.com/paper/2505.22781/) (2025)
- [Solving Continuous Mean Field Games: Deep Reinforcement Learning for Non-Stationary Dynamics](https://mlsanity.com/paper/2510.22158/) (2025)
- [Structural Reinforcement Learning for Heterogeneous Agent Macroeconomics](https://mlsanity.com/paper/2512.18892/) (2025)

## Abstract (arXiv)

Non-cooperative and cooperative games with a very large number of players have many applications but remain generally intractable when the number of players increases. Introduced by Lasry and Lions, and Huang, Caines and Malham'e, Mean Field Games (MFGs) rely on a mean-field approximation to allow the number of players to grow to infinity. Traditional methods for solving these games generally rely on solving partial or stochastic differential equations with a full knowledge of the model. Recently, Reinforcement Learning (RL) has appeared promising to solve complex problems at scale. The combination of RL and MFGs is promising to solve games at a very large scale both in terms of population size and environment complexity. In this survey, we review the quickly growing recent literature on RL methods to learn equilibria and social optima in MFGs. We first identify the most common settings (static, stationary, and evolutive) of MFGs. We then present a general framework for classical iterative methods (based on best-response computation or policy evaluation) to solve MFGs in an exact way. Building on these algorithms and the connection with Markov Decision Processes, we explain how RL can be used to learn MFG solutions in a model-free way. Last, we present numerical illustrations on a benchmark problem, and conclude with some perspectives.