mlsanityData to 7 Oct 2026

On Imitation in Mean-field Games

arXiv 2306.14799 ↗First posted 26 Jun 2023cs.LG cs.GTcited by 10

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

We explore the problem of imitation learning (IL) in the context of mean-field games (MFGs), where the goal is to imitate the behavior of a population of agents following a Nash equilibrium policy according to some unknown payoff function. IL in MFGs presents new challenges compared to single-agent IL, particularly when both the reward function and the transition kernel depend on the population distribution. In this paper, departing from the existing literature on IL for MFGs, we introduce a new solution concept called the Nash imitation gap. Then we show that when only the reward depends on the population distribution, IL in MFGs can be reduced to single-agent IL with similar guarantees. However, when the dynamics is population-dependent, we provide a novel upper-bound that suggests IL is harder in this setting. To address this issue, we propose a new adversarial formulation where the reinforcement learning problem is replaced by a mean-field control (MFC) problem, suggesting progress in IL within MFGs may have to build upon MFC.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 21 of its 38 references that are in the map. First posted on arXiv 26 Jun 2023.

Builds on

21 of 38 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Individual-Level Inverse Reinforcement Learning for Mean Field Games13 Feb
  2. 2021Adversarial Inverse Reinforcement Learning for Mean Field Games29 Apr

Other references · 19

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  3. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 191
  4. 2019A Divergence Minimization Perspective on Imitation Learning Methodscited by 188
  5. 2019Imitation Learning as f-Divergence Minimizationcited by 121
  6. 2020Error Bounds of Imitating Policies and Environmentscited by 90
  7. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gapcited by 80
  8. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 65
  9. 2020Fictitious Play for Mean Field Games: Continuous Time Analysis and Applicationscited by 57
  10. 2019Model-Free Mean-Field Reinforcement Learning: Mean-Field MDP and Mean-Field Q-Learningcited by 56
  11. 2021Concave Utility Reinforcement Learning: the Mean-Field Game Viewpointcited by 54
  12. 2020Mean-Field Controls with Q-learning for Cooperative MARL: Convergence and Complexity Analysiscited by 44
7 more
  1. 2020Unified Reinforcement Q-Learning for Mean Field Game and Control Problemscited by 42
  2. 2020Q-Learning in Regularized Mean-field Gamescited by 40
  3. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  4. 2022Policy Mirror Ascent for Efficient and Independent Learning in Mean Field Gamescited by 24
  5. 2021Deep Learning for Mean Field Games and Mean Field Control with Applications to Financecited by 20
  6. 2018Linearly Solvable Mean-Field Road Traffic Gamescited by 2
  7. 2020Research on Intelligent Traffic Control Methods at Intersections Based on Game Theorycited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 5.

10 papers.

  1. 2024Maximum Causal Entropy IRL in Mean-Field Games and GNEP Framework for Forward RLcited by 4
  2. 2024On Feasible Rewards in Multi-Agent Inverse Reinforcement Learningcited by 4
  3. 2025Learning Equilibria from Data: Provably Efficient Multi-Agent Imitation Learningcited by 4
  4. 2025Learning Surrogate Potential Mean Field Games via Gaussian Processes: A Data-Driven Approach to Ill-Posed Inverse Problemscited by 2
  5. 2025Rate optimal learning of equilibria from datacited by 2
  6. 2026Multi-agent imitation learning with function approximation: Linear Markov games and beyondcited by 1
  7. 2026Matching Multiple Experts: On the Exploitability of Multi-Agent Imitation Learningcited by 0
  8. 2026Population-Aware Imitation Learning in Mean-field Games with Common Noisecited by 0
  9. 2026Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Rewardcited by 0
  10. 2026All in One: Generative Modeling as Mean-Field Game Designcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Mean Field Correlated Imitation Learning
  2. 2025Meta-Inverse Reinforcement Learning for Mean Field Games via Probabilistic Context Variables
  3. 2023Mimicking To Dominate: Imitation Learning Strategies for Success in Multiagent Competitive Games
  4. 2018Multi-Agent Generative Adversarial Imitation Learning
  5. 2024Multi-Agent Imitation Learning: Value is Easy, Regret is Hard
  6. 2024Scalable Offline Reinforcement Learning for Mean Field Games
  7. 2022A Ranking Game for Imitation Learning
  8. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions