# Population-Aware Imitation Learning in Mean-field Games with Common Noise

arXiv: https://arxiv.org/abs/2605.03357 · HTML: https://mlsanity.com/paper/2605.03357/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 17 of its 39 references that are in the map.
- First posted on arXiv 5 May 2026.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 17 of 17)

- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1802.05438/) (2018)
- [Fictitious Play for Mean Field Games: Continuous Time Analysis and Applications](https://mlsanity.com/paper/2007.03458/) (2020)
- [On the Convergence of Model Free Learning in Mean Field Games](https://mlsanity.com/paper/1907.02633/) (2019)
- [Generalization in Mean Field Games by Learning Master Policies](https://mlsanity.com/paper/2109.09717/) (2021)
- [Mean Field Games and Applications: Numerical Aspects](https://mlsanity.com/paper/2003.04444/) (2020)
- [Individual-Level Inverse Reinforcement Learning for Mean Field Games](https://mlsanity.com/paper/2202.06401/) (2022)
- [On Imitation in Mean-field Games](https://mlsanity.com/paper/2306.14799/) (2023)
- [Linear Quadratic Mean Field Type Control and Mean Field Games with Common Noise, with Application to Production of an Exhaustible Resource](https://mlsanity.com/paper/1607.02130/) (2016)
- [Population-aware Online Mirror Descent for Mean-Field Games with Common Noise by Deep Reinforcement Learning](https://mlsanity.com/paper/2509.03030/) (2025)
- [Meta-Inverse Reinforcement Learning for Mean Field Games via Probabilistic Context Variables](https://mlsanity.com/paper/2509.03845/) (2025)
- [Energy transition under scenario uncertainty: a mean-field game of stopping with common noise](https://mlsanity.com/paper/2210.03554/) (2022)
- [Global Solutions to Master Equations for Continuous Time Heterogeneous Agent Macroeconomic Models](https://mlsanity.com/paper/2406.13726/) (2024)
- [A Mean Field Game of Optimal Portfolio Liquidation](https://mlsanity.com/paper/1804.04911/) (2018)
- [Mean Field Game Systems with Common Noise and Markovian Latent Processes](https://mlsanity.com/paper/1809.07865/) (2018)
- [Machine Learning architectures for price formation models with common noise](https://mlsanity.com/paper/2305.17618/) (2023)
- [Mean Field Game of Mutual Holding with common noise](https://mlsanity.com/paper/2403.16232/) (2024)

## Abstract (arXiv)

Mean Field Games (MFGs) provide a powerful framework for modeling the collective behavior of large populations of interacting agents. In this paper, we address the problem of Imitation Learning (IL) in MFGs subject to common noise, where the population distribution evolves stochastically. This stochasticity compels agents to adopt population-aware policies to respond to aggregate shocks. We formulate two distinct learning objectives: recovering a Nash equilibrium and maximizing performance against an expert population. We investigate two imitation proxies: Behavioral Cloning (BC) and Adversarial (ADV) divergence. We then establish finite-sample error bounds showing that minimizing these proxies effectively controls both the policy's exploitability and its performance gap relative to the expert. Furthermore, we propose a numerical framework using generalized Fictitious Play and Deep Learning to compute expert population-aware policies. Through experiments on three environments we demonstrate that standard population-unaware policies fail to capture the equilibrium dynamics. Our results highlight that learning population-aware policies is crucial to avoid being misled by the randomness inherent in common noise.