mlsanityData to 7 Oct 2026

Unified Reinforcement Q-Learning for Mean Field Game and Control Problems

arXiv 2006.13912 ↗First posted 24 Jun 2020math.OC cs.LG cs.MAcited by 42

Not on the map. Why?

Abstract (arXiv)

We present a Reinforcement Learning (RL) algorithm to solve infinite horizon asymptotic Mean Field Game (MFG) and Mean Field Control (MFC) problems. Our approach can be described as a unified two-timescale Mean Field Q-learning: The same algorithm can learn either the MFG or the MFC solution by simply tuning the ratio of two learning parameters. The algorithm is in discrete time and space where the agent not only provides an action to the environment but also a distribution of the state in order to take into account the mean field feature of the problem. Importantly, we assume that the agent can not observe the population's distribution and needs to estimate it in a model-free manner. The asymptotic MFG and MFC problems are also presented in continuous time and space, and compared with classical (non-asymptotic or stationary) MFG and MFC problems. They lead to explicit solutions in the linear-quadratic (LQ) case that are used as benchmarks for the results of our algorithm.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 4; 2023: 12; 2024: 9; 2025: 6; 2026: 6.

42 papers.

Sort:

By area:

  1. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  2. 2021On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC)cited by 31
  3. 2021Scaling up Mean Field Games with Online Mirror Descentcited by 29
  4. 2023Actor-Critic learning for mean-field control in continuous timecited by 19
  5. 2023Recent Developments in Machine Learning Methods for Stochastic Control and Gamescited by 19
  6. 2023Continuous-time q-learning for mean-field control problemscited by 17
  7. 2022Oracle-free Reinforcement Learning in Mean-Field Games along a Single Sample Pathcited by 16
  8. 2021Reinforcement Learning for Mean Field Games, with Applications to Economicscited by 14
  9. 2023On the Statistical Efficiency of Mean-Field Reinforcement Learning with General Function Approximationcited by 13
  10. 2021Signatured Deep Fictitious Play for Mean Field Games with Common Noisecited by 12
  11. 2023On Imitation in Mean-field Gamescited by 10
  12. 2023Mean-Field Control based Approximation of Multi-Agent Reinforcement Learning in Presence of a Non-decomposable Shared Global Statecited by 8
30 more
  1. 2024Unified continuous-time q-learning for mean-field game and mean-field control problemscited by 7
  2. 2022Can Mean Field Control (MFC) Approximate Cooperative Multi Agent Reinforcement Learning (MARL) with Non-Uniform Interaction?cited by 6
  3. 2023Actor critic learning algorithms for mean-field control with moment neural networkscited by 6
  4. 2023Deep Reinforcement Learning for Infinite Horizon Mean Field Problems in Continuous Spacescited by 6
  5. 2023Efficient Model-Based Concave Utility Reinforcement Learning through Greedy Mirror Descentcited by 6
  6. 2022Mean-Field Approximation of Cooperative Constrained Multi-Agent Reinforcement Learning (CMARL)cited by 5
  7. 2023Regularization of the policy updates for stabilizing Mean Field Gamescited by 5
  8. 2024MF-OML: Online Mean-Field Reinforcement Learning with Occupation Measures for Large Population Gamescited by 5
  9. 2024Graphon Mean Field Games with a Representative Player: Analysis and Learning Algorithmcited by 5
  10. 2024Stochastic Semi-Gradient Descent for Learning Mean Field Games with Population-Aware Function Approximationcited by 5
  11. 2022On the Near-Optimality of Local Policies in Large Cooperative Multi-Agent Reinforcement Learningcited by 4
  12. 2023Safe Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 3
  13. 2024Analysis of Multiscale Reinforcement Q-Learning Algorithms for Mean Field Control Gamescited by 3
  14. 2025Convergence of Actor-Critic Learning for Mean Field Games and Mean Field Control in Continuous Spacescited by 3
  15. 2026Model-free policy gradient for discrete-time mean-field controlcited by 3
  16. 2026Continuous-time q-learning for mean-field control with common noise, part-II: q-learning algorithmscited by 3
  17. 2024Independent RL for Cooperative-Competitive Agents: A Mean-Field Perspectivecited by 2
  18. 2024Policy Optimization finds Nash Equilibrium in Regularized General-Sum LQ Gamescited by 2
  19. 2024A Single Online Agent Can Efficiently Learn Mean Field Gamescited by 2
  20. 2025Convergence of two-timescale gradient descent ascent dynamics: finite-dimensional and mean-field perspectivescited by 2
  21. 2025Finite-Sample Convergence Bounds for Trust Region Policy Optimization in Mean-Field Gamescited by 2
  22. 2025Solving Continuous Mean Field Games: Deep Reinforcement Learning for Non-Stationary Dynamicscited by 2
  23. 2025Policy Optimization for Continuous-time Linear-Quadratic Graphon Mean Field Gamescited by 1
  24. 2026Bench-MFG: A Benchmark Suite for Learning in Stationary Mean Field Gamescited by 1
  25. 2026Operator-Theoretic Foundations and Policy Gradient Methods for General MDPs with Unbounded Costscited by 1
  26. 2023Networked Communication for Decentralised Agents in Mean-Field Gamescited by 0
  27. 2024Last Iterate Convergence in Monotone Mean Field Gamescited by 0
  28. 2025Vulnerable Agent Identification in Large-Scale Multi-Agent Reinforcement Learningcited by 0
  29. 2026Stationary Robust Mean-Field Games under Model Mismatchescited by 0
  30. 2026Mean-Field PhiBE: Continuous-Time Mean-Field Reinforcement Learning from Discrete-Time Datacited by 0