# Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market

arXiv: https://arxiv.org/abs/2112.04494 · HTML: https://mlsanity.com/paper/2112.04494/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 18 references that are in the map.
- First posted on arXiv 8 Dec 2021.

## Builds on (most cited first, 5 of 5)

- [A Survey of Deep Reinforcement Learning in Video Games](https://mlsanity.com/paper/1912.10944/) (2019)
- [Market Making via Reinforcement Learning](https://mlsanity.com/paper/1804.04216/) (2018)
- [Reinforcement Learning for Market Making in a Multi-agent Dealer Market](https://mlsanity.com/paper/1911.05892/) (2019)
- [ABIDES: Towards High-Fidelity Market Simulation for AI Research](https://mlsanity.com/paper/1904.12066/) (2019)
- [Optimizing Market Making using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1812.10252/) (2018)

## Built on by (most cited first, 1 of 1)

- [Reinforcement Learning with Hawkes-Structured Networks for Market Making](https://mlsanity.com/paper/2610.16524/) (2026)

## Abstract (arXiv)

Market makers play a key role in financial markets by providing liquidity. They usually fill order books with buy and sell limit orders in order to provide traders alternative price levels to operate. This paper focuses precisely on the study of these markets makers strategies from an agent-based perspective. In particular, we propose the application of Reinforcement Learning (RL) for the creation of intelligent market markers in simulated stock markets. This research analyzes how RL market maker agents behaves in non-competitive (only one RL market maker learning at the same time) and competitive scenarios (multiple RL market markers learning at the same time), and how they adapt their strategies in a Sim2Real scope with interesting results. Furthermore, it covers the application of policy transfer between different experiments, describing the impact of competing environments on RL agents performance. RL and deep RL techniques are proven as profitable market maker approaches, leading to a better understanding of their behavior in stock markets.