# Deep Q-Learning for Nash Equilibria: Nash-DQN

arXiv: https://arxiv.org/abs/1904.10554 · HTML: https://mlsanity.com/paper/1904.10554/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2019.

## Built on by (most cited first, 14 of 14)

- [Recent Developments in Machine Learning Methods for Stochastic Control and Games](https://mlsanity.com/paper/2303.10257/) (2023)
- [Reinforcement Learning with Dynamic Convex Risk Measures](https://mlsanity.com/paper/2112.13414/) (2021)
- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [Incentivizing Truthful Language Models via Peer Elicitation Games](https://mlsanity.com/paper/2505.13636/) (2025)
- [Reinforcement Learning for Finite Space Mean-Field Type Games](https://mlsanity.com/paper/2409.18152/) (2024)
- [Scalable, Decentralized Multi-Agent Reinforcement Learning Methods Inspired by Stigmergy and Ant Colonies](https://mlsanity.com/paper/2105.03546/) (2021)
- [Multi-Agent Reinforcement Learning for Greenhouse Gas Offset Credit Markets](https://mlsanity.com/paper/2504.11258/) (2025)
- [Deep reinforcement learning for optimal trading with partial information](https://mlsanity.com/paper/2511.00190/) (2025)
- [Trading Signals In VIX Futures](https://mlsanity.com/paper/2103.02016/) (2021)
- [Learning Strategic Value and Cooperation in Multi-Player Stochastic Games through Side Payments](https://mlsanity.com/paper/2303.05307/) (2023)
- [Asynchronous Fractional Multi-Agent Deep Reinforcement Learning for Age-Minimal Mobile Edge Computing](https://mlsanity.com/paper/2409.16832/) (2024)
- [Learning with Dynamics: Autonomous Regulation of UAV Based Communication Networks with Dynamic UAV Crew](https://mlsanity.com/paper/2409.17139/) (2024)
- [Generative Market Equilibrium Models with Stable Adversarial Learning via Reinforcement](https://mlsanity.com/paper/2504.04300/) (2025)
- [DNQ: Deep Nash Q-Network for Partially Observable n-Player Games](https://mlsanity.com/paper/2606.06480/) (2026)

## Abstract (arXiv)

Model-free learning for multi-agent stochastic games is an active area of research. Existing reinforcement learning algorithms, however, are often restricted to zero-sum games, and are applicable only in small state-action spaces or other simplified settings. Here, we develop a new data efficient Deep-Q-learning methodology for model-free learning of Nash equilibria for general-sum stochastic games. The algorithm uses a local linear-quadratic expansion of the stochastic game, which leads to analytically solvable optimal actions. The expansion is parametrized by deep neural networks to give it sufficient flexibility to learn the environment without the need to experience all state-action pairs. We study symmetry properties of the algorithm stemming from label-invariant stochastic games and as a proof of concept, apply our algorithm to learning optimal trading strategies in competitive electronic markets.