mlsanityData to 7 Oct 2026

Scaling up Mean Field Games with Online Mirror Descent

arXiv 2103.00623 ↗First posted 28 Feb 2021cs.AIcited by 29

Not on the map. Why?

Abstract (arXiv)

We address scaling up equilibrium computation in Mean Field Games (MFGs) using Online Mirror Descent (OMD). We show that continuous-time OMD provably converges to a Nash equilibrium under a natural and well-motivated set of monotonicity assumptions. This theoretical result nicely extends to multi-population games and to settings involving common noise. A thorough experimental investigation on various single and multi-population MFGs shows that OMD outperforms traditional algorithms such as Fictitious Play (FP). We empirically show that OMD scales up and converges significantly faster than FP by solving, for the first time to our knowledge, examples of MFGs with hundreds of billions states. This study establishes the state-of-the-art for learning in large-scale multi-agent and multi-population games.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 27 of its 82 references that are in the map. First posted on arXiv 28 Feb 2021.

Builds on

27 of 82 references.

References · 27

  1. 2017DeepStack: Expert-Level Artificial Intelligence in No-Limit Pokercited by 228
  2. 2017Cycles in adversarial regularized learningcited by 122
  3. 2019A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problemscited by 95
  4. 2020Fictitious Play for Mean Field Games: Continuous Time Analysis and Applicationscited by 57
  5. 2019On the Convergence of Model Free Learning in Mean Field Gamescited by 45
  6. 2020Unified Reinforcement Q-Learning for Mean Field Game and Control Problemscited by 42
  7. 2020Q-Learning in Regularized Mean-field Gamescited by 40
  8. 2015Learning in Mean Field Games: the Fictitious Playcited by 37
  9. 2020Adapting a Kidney Exchange Algorithm to Align with Human Valuescited by 36
  10. 2019Convergence Analysis of Machine Learning Algorithms for the Numerical Solution of Mean Field Control and Games: II -- The Finite Horizon Casecited by 34
  11. 2018Bandit learning in concave N-person gamescited by 32
  12. 2017Multiplicative Weights Update with Constant Step-Size in Congestion Games: Convergence, Limit Cycles and Chaoscited by 29
15 more
  1. 2019Deep Learning Methods for Mean Field Control Problems with Delaycited by 24
  2. 2020Mean Field Games and Applications: Numerical Aspectscited by 22
  3. 2019Numerical resolution of McKean-Vlasov FBSDEs using neural networkscited by 19
  4. 2019The route to chaos in routing games: When is Price of Anarchy too optimistic?cited by 14
  5. 2017Learning in anonymous nonatomic games with applications to first-order mean field gamescited by 10
  6. 2016Proximal methods for stationary Mean Field Games with local couplingscited by 9
  7. 2018On the implementation of a primal-dual algorithm for second order time-dependent mean field games with local couplingscited by 9
  8. 2018Finite mean field games: fictitious play and convergence to a first order continuous mean field gamecited by 9
  9. 2020Connecting GANs, MFGs, and OTcited by 8
  10. 2020Model-free Reinforcement Learning for Non-stationary Mean Field Gamescited by 7
  11. 2020Provable Fictitious Play for General Mean-Field Gamescited by 7
  12. 2018Mean Field Control and Mean Field Game Models with Several Populationscited by 5
  13. 2015On the system of partial differential equations arising in mean field type controlcited by 4
  14. 2017Mean-field type modeling of nonlocal crowd aversion in pedestrian crowd dynamicscited by 4
  15. 2020Convergence of some Mean Field Games systems to aggregation and flocking modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 5; 2023: 9; 2024: 7; 2025: 2; 2026: 2.

29 papers.

Sort:

By area:

  1. 2021Concave Utility Reinforcement Learning: the Mean-Field Game Viewpointcited by 54
  2. 2022Scalable Deep Reinforcement Learning Algorithms for Mean Field Gamescited by 33
  3. 2021Generalization in Mean Field Games by Learning Master Policiescited by 28
  4. 2021Learning Graphon Mean Field Games and Approximate Nash Equilibriacited by 28
  5. 2021Reinforcement Learning for Mean Field Games, with Applications to Economicscited by 14
  6. 2022Learning Sparse Graphon Mean Field Gamescited by 14
  7. 2023On the Statistical Efficiency of Mean-Field Reinforcement Learning with General Function Approximationcited by 13
  8. 2024Model-Based RL for Mean-Field Games is not Statistically Harder than Single-Agent RLcited by 9
  9. 2023Learning Regularized Monotone Graphon Mean-Field Gamescited by 8
  10. 2024Unified continuous-time q-learning for mean-field game and mean-field control problemscited by 7
  11. 2023Regularization of the policy updates for stabilizing Mean Field Gamescited by 5
  12. 2024MF-OML: Online Mean-Field Reinforcement Learning with Occupation Measures for Large Population Gamescited by 5
17 more
  1. 2024Stochastic Semi-Gradient Descent for Learning Mean Field Games with Population-Aware Function Approximationcited by 5
  2. 2022Learning Correlated Equilibria in Mean-Field Gamescited by 4
  3. 2023Learning Regularized Graphon Mean-Field Games with Unknown Graphonscited by 4
  4. 2024On Feasible Rewards in Multi-Agent Inverse Reinforcement Learningcited by 4
  5. 2022Hypergraphon Mean Field Gamescited by 3
  6. 2023Population-size-Aware Policy Optimization for Mean-Field Gamescited by 3
  7. 2022Developing, Evaluating and Scaling Learning Agents in Multi-Agent Environmentscited by 2
  8. 2024A Single Online Agent Can Efficiently Learn Mean Field Gamescited by 2
  9. 2023MESOB: Balancing Equilibria & Social Optimalitycited by 1
  10. 2023Reinforcement Learning for SBM Graphon Games with Re-Samplingcited by 1
  11. 2025Steering No-Regret Agents in MFGs under Model Uncertaintycited by 1
  12. 2023Reimagining Demand-Side Management with Mean Field Learningcited by 0
  13. 2023Networked Communication for Decentralised Agents in Mean-Field Gamescited by 0
  14. 2024Networked Communication for Mean-Field Games with Function Approximation and Empirical Mean-Field Estimationcited by 0
  15. 2025Vulnerable Agent Identification in Large-Scale Multi-Agent Reinforcement Learningcited by 0
  16. 2026Recurrent Structural Policy Gradient for Partially Observable Mean Field Gamescited by 0
  17. 2026Towards Model-Free Learning in Dynamic Population Games: An Application to Karma Economiescited by 0