# Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games

arXiv: https://arxiv.org/abs/1912.01809 · HTML: https://mlsanity.com/paper/1912.01809/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2019.

## Built on by (most cited first, 16 of 16)

- [Stochastic Optimal Control Matching](https://mlsanity.com/paper/2312.02027/) (2023)
- [Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomization](https://mlsanity.com/paper/2103.04564/) (2021)
- [Recent Developments in Machine Learning Methods for Stochastic Control and Games](https://mlsanity.com/paper/2303.10257/) (2023)
- [Learning High-Dimensional McKean-Vlasov Forward-Backward Stochastic Differential Equations with General Distribution Dependence](https://mlsanity.com/paper/2204.11924/) (2022)
- [Signatured Deep Fictitious Play for Mean Field Games with Common Noise](https://mlsanity.com/paper/2106.03272/) (2021)
- [DeepHAM: A Global Solution Method for Heterogeneous Agent Models with Aggregate Shocks](https://mlsanity.com/paper/2112.14377/) (2021)
- [Amortized Control of Continuous State Space Feynman-Kac Model for Irregular Time Series](https://mlsanity.com/paper/2410.05602/) (2024)
- [KnowRU: Knowledge Reusing via Knowledge Distillation in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2103.14891/) (2021)
- [A Class of Dimension-free Metrics for the Convergence of Empirical Measures](https://mlsanity.com/paper/2104.12036/) (2021)
- [Deep Learning for Principal-Agent Mean Field Games](https://mlsanity.com/paper/2110.01127/) (2021)
- [A brief review of the Deep BSDE method for solving high-dimensional partial differential equations](https://mlsanity.com/paper/2505.17032/) (2025)
- [Recurrent Neural Networks for Stochastic Control Problems with Delay](https://mlsanity.com/paper/2101.01385/) (2021)
- [Deep Graphic FBSDEs for Opinion Dynamics Stochastic Control](https://mlsanity.com/paper/2204.02506/) (2022)
- [Finite-Agent Stochastic Differential Games on Large Graphs: II. Graph-Based Architectures](https://mlsanity.com/paper/2509.12484/) (2025)
- [Pandemic Control, Game Theory and Machine Learning](https://mlsanity.com/paper/2208.08646/) (2022)
- [Stochastic Delay Differential Games: Financial Modeling and Machine Learning Algorithms](https://mlsanity.com/paper/2307.06450/) (2023)

## Abstract (arXiv)

We propose a deep neural network-based algorithm to identify the Markovian Nash equilibrium of general large N-player stochastic differential games. Following the idea of fictitious play, we recast the N-player game into N decoupled decision problems (one for each player) and solve them iteratively. The individual decision problem is characterized by a semilinear Hamilton-Jacobi-Bellman equation, to solve which we employ the recently developed deep BSDE method. The resulted algorithm can solve large N-player games for which conventional numerical methods would suffer from the curse of dimensionality. Multiple numerical examples involving identical or heterogeneous agents, with risk-neutral or risk-sensitive objectives, are tested to validate the accuracy of the proposed algorithm in large group games. Even for a fifty-player game with the presence of common noise, the proposed algorithm still finds the approximate Nash equilibrium accurately, which, to our best knowledge, is difficult to achieve by other numerical algorithms.