mlsanityData to 7 Oct 2026

Learning Regularized Monotone Graphon Mean-Field Games

arXiv 2310.08089 ↗First posted 12 Oct 2023cs.GT cs.SY eess.SYcited by 8

Not on the map. Why?

Abstract (arXiv)

This paper studies two fundamental problems in regularized Graphon Mean-Field Games (GMFGs). First, we establish the existence of a Nash Equilibrium (NE) of any λ-regularized GMFG (for λ≥ 0). This result relies on weaker conditions than those in previous works for analyzing both unregularized GMFGs (λ=0) and λ-regularized MFGs, which are special cases of GMFGs. Second, we propose provably efficient algorithms to learn the NE in weakly monotone GMFGs, motivated by Lasry and Lions [2007]. Previous literature either only analyzed continuous-time algorithms or required extra conditions to analyze discrete-time algorithms. In contrast, we design a discrete-time algorithm and derive its convergence rate solely under weakly monotone conditions. Furthermore, we develop and analyze the action-value function estimation procedure during the online learning process, which is absent from algorithms for monotone GMFGs. This serves as a sub-module in our optimization algorithm. The efficiency of the designed algorithm is corroborated by empirical evaluations.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 23 of its 44 references that are in the map. First posted on arXiv 12 Oct 2023.

Builds on

23 of 44 references.

References · 23

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  2. 2019A Theory of Regularized Markov Decision Processescited by 279
  3. 2021Bellman-consistent Pessimism for Offline Reinforcement Learningcited by 267
  4. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 257
  5. 2019Provably Efficient Exploration in Policy Optimizationcited by 203
  6. 2020Fast Global Convergence of Natural Policy Gradient Methods with Entropy Regularizationcited by 170
  7. 2019Adaptive Trust Region Policy Optimization: Global Convergence and Faster Rates for Regularized MDPscited by 113
  8. 2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 92
  9. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 65
  10. 2020Fictitious Play for Mean Field Games: Continuous Time Analysis and Applicationscited by 57
  11. 2021Concave Utility Reinforcement Learning: the Mean-Field Game Viewpointcited by 54
  12. 2020Q-Learning in Regularized Mean-field Gamescited by 40
11 more
  1. 2021Scaling up Mean Field Games with Online Mirror Descentcited by 29
  2. 2021Learning Graphon Mean Field Games and Approximate Nash Equilibriacited by 28
  3. 2022Policy Mirror Ascent for Efficient and Independent Learning in Mean Field Gamescited by 24
  4. 2020Mean Field Games and Applications: Numerical Aspectscited by 22
  5. 2020Breaking the Curse of Many Agents: Provable Mean Embedding Q-Iteration for Mean-Field Reinforcement Learningcited by 19
  6. 2022Oracle-free Reinforcement Learning in Mean-Field Games along a Single Sample Pathcited by 16
  7. 2022Learning Sparse Graphon Mean Field Gamescited by 14
  8. 2019Stochastic Graphon Games: I. The Static Casecited by 12
  9. 2021Finite State Graphon Games with Applications to Epidemicscited by 9
  10. 2018Graphon games: A statistical framework for network games and interventionscited by 8
  11. 2020Master equation of discrete time graphon mean field games and teamscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 1.

8 papers.

  1. 2024Model-Based RL for Mean-Field Games is not Statistically Harder than Single-Agent RLcited by 9
  2. 2024MF-OML: Online Mean-Field Reinforcement Learning with Occupation Measures for Large Population Gamescited by 5
  3. 2024Graphon Mean Field Games with a Representative Player: Analysis and Learning Algorithmcited by 5
  4. 2025Learning Mean Field Control on Sparse Graphscited by 2
  5. 2025Convergence and Connectivity: Dynamics of Multi-Agent Q-Learning in Random Networkscited by 1
  6. 2025Policy Optimization for Continuous-time Linear-Quadratic Graphon Mean Field Gamescited by 1
  7. 2024Last Iterate Convergence in Monotone Mean Field Gamescited by 0
  8. 2026Mean-Field Control on Sparse Graphs: From Local Limits to GNNs via Neighborhood Distributionscited by 0