Population-Aware Imitation Learning in Mean-field Games with Common Noise
On the map: Offline RL and imitation learning from data → Inverse reinforcement learning
Abstract (arXiv)
Mean Field Games (MFGs) provide a powerful framework for modeling the collective behavior of large populations of interacting agents. In this paper, we address the problem of Imitation Learning (IL) in MFGs subject to common noise, where the population distribution evolves stochastically. This stochasticity compels agents to adopt population-aware policies to respond to aggregate shocks. We formulate two distinct learning objectives: recovering a Nash equilibrium and maximizing performance against an expert population. We investigate two imitation proxies: Behavioral Cloning (BC) and Adversarial (ADV) divergence. We then establish finite-sample error bounds showing that minimizing these proxies effectively controls both the policy's exploitability and its performance gap relative to the expert. Furthermore, we propose a numerical framework using generalized Fictitious Play and Deep Learning to compute expert population-aware policies. Through experiments on three environments we demonstrate that standard population-unaware policies fail to capture the equilibrium dynamics. Our results highlight that learning population-aware policies is crucial to avoid being misled by the randomness inherent in common noise.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 17 of its 39 references that are in the map. First posted on arXiv 5 May 2026.
Builds on
17 of 39 references.
Closest ancestors · 3
Same direction, newest first
- 2025Meta-Inverse Reinforcement Learning for Mean Field Games via Probabilistic Context Variables4 Sep
- 2023On Imitation in Mean-field Games26 Jun
- 2022Individual-Level Inverse Reinforcement Learning for Mean Field Games13 Feb
Other references · 14
- 2016Generative Adversarial Imitation Learningcited by 1,576
- 2018Mean Field Multi-Agent Reinforcement Learningcited by 202
- 2020Fictitious Play for Mean Field Games: Continuous Time Analysis and Applicationscited by 57
- 2019On the Convergence of Model Free Learning in Mean Field Gamescited by 45
- 2021Generalization in Mean Field Games by Learning Master Policiescited by 28
- 2020Mean Field Games and Applications: Numerical Aspectscited by 22
- 2016Linear Quadratic Mean Field Type Control and Mean Field Games with Common Noise, with Application to Production of an Exhaustible Resourcecited by 6
- 2025Population-aware Online Mirror Descent for Mean-Field Games with Common Noise by Deep Reinforcement Learningcited by 4
- 2024Global Solutions to Master Equations for Continuous Time Heterogeneous Agent Macroeconomic Modelscited by 2
- 2018A Mean Field Game of Optimal Portfolio Liquidationcited by 1
- 2018Mean Field Game Systems with Common Noise and Markovian Latent Processescited by 1
- 2022Energy transition under scenario uncertainty: a mean-field game of stopping with common noisecited by 1
2 more
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Matching Multiple Experts: On the Exploitability of Multi-Agent Imitation Learning
- 2024Mean Field Correlated Imitation Learning
- 2023Mimicking To Dominate: Imitation Learning Strategies for Success in Multiagent Competitive Games
- 2025Learning Equilibria from Data: Provably Efficient Multi-Agent Imitation Learning
- 2021Adversarial Inverse Reinforcement Learning for Mean Field Games
- 2026Mean-Field Diffuser: Scaling Offline MARL to Thousands of Agents
- 2024Scalable Offline Reinforcement Learning for Mean Field Games
- 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap