# Solving Continuous Mean Field Games: Deep Reinforcement Learning for Non-Stationary Dynamics

arXiv: https://arxiv.org/abs/2510.22158 · HTML: https://mlsanity.com/paper/2510.22158/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 31 of its 52 references that are in the map.
- First posted on arXiv 25 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 30 of 31)

- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Learning Likelihoods with Conditional Normalizing Flows](https://mlsanity.com/paper/1912.00042/) (2019)
- [Deep Reinforcement Learning from Self-Play in Imperfect-Information Games](https://mlsanity.com/paper/1603.01121/) (2016)
- [Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning](https://mlsanity.com/paper/2206.15378/) (2022)
- [Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learning](https://mlsanity.com/paper/2102.01585/) (2021)
- [Fictitious Play for Mean Field Games: Continuous Time Analysis and Applications](https://mlsanity.com/paper/2007.03458/) (2020)
- [Learning in Mean Field Games: A Survey](https://mlsanity.com/paper/2205.12944/) (2022)
- [Concave Utility Reinforcement Learning: the Mean-Field Game Viewpoint](https://mlsanity.com/paper/2106.03787/) (2021)
- [Unified Reinforcement Q-Learning for Mean Field Game and Control Problems](https://mlsanity.com/paper/2006.13912/) (2020)
- [On the Convergence of Model Free Learning in Mean Field Games](https://mlsanity.com/paper/1907.02633/) (2019)
- [Deep Multiagent Reinforcement Learning: Challenges and Directions](https://mlsanity.com/paper/2106.15691/) (2021)
- [Learning in Mean Field Games: the Fictitious Play](https://mlsanity.com/paper/1507.06280/) (2015)
- [Scalable Deep Reinforcement Learning Algorithms for Mean Field Games](https://mlsanity.com/paper/2203.11973/) (2022)
- [Mean Field Games Flock! The Reinforcement Learning Way](https://mlsanity.com/paper/2105.07933/) (2021)
- [Policy Mirror Ascent for Efficient and Independent Learning in Mean Field Games](https://mlsanity.com/paper/2212.14449/) (2022)
- [A General Framework for Learning Mean-Field Games](https://mlsanity.com/paper/2003.06069/) (2020)
- [Recent Developments in Machine Learning Methods for Stochastic Control and Games](https://mlsanity.com/paper/2303.10257/) (2023)
- [Mean Field Game of Controls and An Application To Trade Crowding](https://mlsanity.com/paper/1610.09904/) (2016)
- [Mean Field Games and Applications: Numerical Aspects](https://mlsanity.com/paper/2003.04444/) (2020)
- [Actor-Critic Provably Finds Nash Equilibria of Linear-Quadratic Mean-Field Games](https://mlsanity.com/paper/1910.07498/) (2019)
- [Mean-Field-Type Games in Engineering](https://mlsanity.com/paper/1605.03281/) (2016)
- [Reinforcement Learning in Non-Stationary Discrete-Time Linear-Quadratic Mean-Field Games](https://mlsanity.com/paper/2009.04350/) (2020)
- [Mean-Field Optimal Control](https://mlsanity.com/paper/1306.5913/) (2013)
- [Efficiency of the Price Formation Process in Presence of High Frequency Participants: a Mean Field Game analysis](https://mlsanity.com/paper/1305.6323/) (2013)
- [Deep Reinforcement Learning for Infinite Horizon Mean Field Problems in Continuous Spaces](https://mlsanity.com/paper/2309.10953/) (2023)
- [Finite-Sample Convergence Bounds for Trust Region Policy Optimization in Mean-Field Games](https://mlsanity.com/paper/2505.22781/) (2025)

## Built on by (most cited first, 2 of 2)

- [Convergence of Actor-Critic Learning for Mean Field Games and Mean Field Control in Continuous Spaces](https://mlsanity.com/paper/2511.06812/) (2025)
- [Bench-MFG: A Benchmark Suite for Learning in Stationary Mean Field Games](https://mlsanity.com/paper/2602.12517/) (2026)

## Abstract (arXiv)

Mean field games (MFGs) have emerged as a powerful framework for modeling interactions in large-scale multi-agent systems. Despite recent advancements in reinforcement learning (RL) for MFGs, existing methods are typically limited to finite spaces or stationary models, hindering their applicability to real-world problems. This paper introduces a novel deep reinforcement learning (DRL) algorithm specifically designed for non-stationary continuous MFGs. The proposed approach builds upon a Fictitious Play (FP) methodology, leveraging DRL for best-response computation and supervised learning for average policy representation. Furthermore, it learns a representation of the time-dependent population distribution using a Conditional Normalizing Flow. To validate the effectiveness of our method, we evaluate it on three different examples of increasing complexity. By addressing critical limitations in scalability and density approximation, this work represents a significant advancement in applying DRL techniques to complex MFG problems, bringing the field closer to real-world multi-agent systems.