mlsanityData to 7 Oct 2026

Population-aware Online Mirror Descent for Mean-Field Games by Deep Reinforcement Learning

arXiv 2403.03552 ↗First posted 6 Mar 2024cs.GT cs.LG cs.MAcited by 5

Not on the map. Why?

Abstract (arXiv)

Mean Field Games (MFGs) have the ability to handle large-scale multi-agent systems, but learning Nash equilibria in MFGs remains a challenging task. In this paper, we propose a deep reinforcement learning (DRL) algorithm that achieves population-dependent Nash equilibrium without the need for averaging or sampling from history, inspired by Munchausen RL and Online Mirror Descent. Through the design of an additional inner-loop replay buffer, the agents can effectively learn to achieve Nash equilibrium from any distribution, mitigating catastrophic forgetting. The resulting policy can be applied to various initial distributions. Numerical experiments on four canonical examples demonstrate our algorithm has better convergence properties than SOTA algorithms, in particular a DRL version of Fictitious Play for population-dependent policies.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 19 of its 40 references that are in the map. First posted on arXiv 6 Mar 2024.

Builds on

19 of 40 references.

References · 19

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  3. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  4. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  5. 2017Time Limits in Reinforcement Learningcited by 72
  6. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 65
  7. 2019Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learningcited by 57
  8. 2020Fictitious Play for Mean Field Games: Continuous Time Analysis and Applicationscited by 57
  9. 2022Learning in Mean Field Games: A Surveycited by 56
  10. 2021Concave Utility Reinforcement Learning: the Mean-Field Game Viewpointcited by 54
  11. 2020Q-Learning in Regularized Mean-field Gamescited by 40
  12. 2015Learning in Mean Field Games: the Fictitious Playcited by 37
7 more
  1. 2021Training Larger Networks for Deep Reinforcement Learningcited by 37
  2. 2022Scalable Deep Reinforcement Learning Algorithms for Mean Field Gamescited by 33
  3. 2019Computing Approximate Equilibria in Sequential Adversarial Games by Exploitability Descentcited by 31
  4. 2021Generalization in Mean Field Games by Learning Master Policiescited by 28
  5. 2017Learning in anonymous nonatomic games with applications to first-order mean field gamescited by 10
  6. 2018Finite mean field games: fictitious play and convergence to a first order continuous mean field gamecited by 9
  7. 2014Linear-Quadratic Mean Field Gamescited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2025Population-aware Online Mirror Descent for Mean-Field Games with Common Noise by Deep Reinforcement Learningcited by 4
  2. 2025Convergence and Connectivity: Dynamics of Multi-Agent Q-Learning in Random Networkscited by 1
  3. 2024Networked Communication for Mean-Field Games with Function Approximation and Empirical Mean-Field Estimationcited by 0
  4. 2026Recurrent Structural Policy Gradient for Partially Observable Mean Field Gamescited by 0
  5. 2026Towards Model-Free Learning in Dynamic Population Games: An Application to Karma Economiescited by 0