mlsanityData to 7 Oct 2026

Individual-Level Inverse Reinforcement Learning for Mean Field Games

arXiv 2202.06401 ↗First posted 13 Feb 2022cs.LG cs.GT cs.MAcited by 10

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

The recent mean field game (MFG) formalism has enabled the application of inverse reinforcement learning (IRL) methods in large-scale multi-agent systems, with the goal of inferring reward signals that can explain demonstrated behaviours of large populations. The existing IRL methods for MFGs are built upon reducing an MFG to a Markov decision process (MDP) defined on the collective behaviours and average rewards of the population. However, this paper reveals that the reduction from MFG to MDP holds only for the fully cooperative setting. This limitation invalidates existing IRL methods on MFGs with non-cooperative environments. To measure more general behaviours in large populations, we study the use of individual behaviours to infer ground-truth reward functions for MFGs. We propose Mean Field IRL (MFIRL), the first dedicated IRL framework for MFGs that can handle both cooperative and non-cooperative environments. Based on this theoretically justified framework, we develop a practical algorithm effective for MFGs with unknown dynamics. We evaluate MFIRL on both cooperative and mixed cooperative-competitive scenarios with many agents. Results demonstrate that MFIRL excels in reward recovery, sample efficiency and robustness in the face of changing dynamics.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 22 of its 58 references that are in the map. First posted on arXiv 13 Feb 2022.

Builds on

22 of 58 references.

Closest ancestors · 1

Same direction, newest first

  1. 2019Multi-Agent Adversarial Inverse Reinforcement Learning30 Jul

Other references · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  6. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  7. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  8. 2018Multi-Agent Generative Adversarial Imitation Learningcited by 99
  9. 2017MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligencecited by 69
  10. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 65
  11. 2019Model-Free Mean-Field Reinforcement Learning: Mean-Field MDP and Mean-Field Q-Learningcited by 56
  12. 2019On the Convergence of Model Free Learning in Mean Field Gamescited by 45
9 more
  1. 2016Markov-Nash Equilibria in Mean-Field Games with Discounted Costcited by 29
  2. 2020Multi Type Mean Field Reinforcement Learningcited by 25
  3. 2013Computational Rationalization: The Inverse Equilibrium Problemcited by 24
  4. 2016Mean Field Game of Controls and An Application To Trade Crowdingcited by 22
  5. 2018Decentralised Learning in Systems with Many, Many Strategic Agentscited by 19
  6. 2016Inverse Reinforcement Learning in Swarm Systemscited by 18
  7. 2020LUCIDGames: Online Unscented Inverse Dynamic Games for Adaptive Trajectory Prediction and Planningcited by 17
  8. 2021Inferring Objectives in Continuous Dynamic Games from Noise-Corrupted Partial State Observationscited by 13
  9. 2013Mean field games with nonlinear mobilities in pedestrian dynamicscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 2; 2025: 2; 2026: 4.

10 papers.

  1. 2022A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learningcited by 12
  2. 2023On Imitation in Mean-field Gamescited by 10
  3. 2024Maximum Causal Entropy IRL in Mean-Field Games and GNEP Framework for Forward RLcited by 4
  4. 2025Kernel Based Maximum Entropy Inverse Reinforcement Learning for Mean-Field Gamescited by 3
  5. 2024Inverse Concave-Utility Reinforcement Learning is Inverse Game Theorycited by 2
  6. 2025Learning Surrogate Potential Mean Field Games via Gaussian Processes: A Data-Driven Approach to Ill-Posed Inverse Problemscited by 2
  7. 2026Population-Aware Imitation Learning in Mean-field Games with Common Noisecited by 0
  8. 2026Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Rewardcited by 0
  9. 2026Neural Parameter Calibration for Finite-State Mean Field Gamescited by 0
  10. 2026Inverse Learning of the Altruism and Cost Level in Mixed-Individual Mean Field Gamescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Adversarial Inverse Reinforcement Learning for Mean Field Games
  2. 2025Meta-Inverse Reinforcement Learning for Mean Field Games via Probabilistic Context Variables
  3. 2023Discovering Individual Rewards in Collective Behavior through Inverse Multi-Agent Reinforcement Learning
  4. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
  5. 2024Scalable Offline Reinforcement Learning for Mean Field Games
  6. 2024On Feasible Rewards in Multi-Agent Inverse Reinforcement Learning
  7. 2018Multi-agent Inverse Reinforcement Learning for Certain General-sum Stochastic Games
  8. 2021Concave Utility Reinforcement Learning: the Mean-Field Game Viewpoint