# Towards a fully RL-based Market Simulator

arXiv: https://arxiv.org/abs/2110.06829 · HTML: https://mlsanity.com/paper/2110.06829/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 21 references that are in the map.
- First posted on arXiv 13 Oct 2021.

## Builds on (most cited first, 5 of 5)

- [The Complexity of Decentralized Control of Markov Decision Processes](https://mlsanity.com/paper/1301.3836/) (2013)
- [Get Real: Realism Metrics for Robust Limit Order Book Market Simulations](https://mlsanity.com/paper/1912.04941/) (2019)
- [Reinforcement Learning for Market Making in a Multi-agent Dealer Market](https://mlsanity.com/paper/1911.05892/) (2019)
- [Calibration of Shared Equilibria in General Sum Partially Observable Markov Games](https://mlsanity.com/paper/2006.13085/) (2020)
- [Agent Environment Cycle Games](https://mlsanity.com/paper/2009.13051/) (2020)

## Built on by (most cited first, 10 of 10)

- [Dynamic Datasets and Market Environments for Financial Reinforcement Learning](https://mlsanity.com/paper/2304.13174/) (2023)
- [Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulations](https://mlsanity.com/paper/2210.07184/) (2022)
- [Phantom -- A RL-driven multi-agent framework to model complex systems](https://mlsanity.com/paper/2210.06012/) (2022)
- [Reinforcement Learning for Combining Search Methods in the Calibration of Economic ABMs](https://mlsanity.com/paper/2302.11835/) (2023)
- [CNN-DRL with Shuffled Features in Finance](https://mlsanity.com/paper/2402.03338/) (2024)
- [Reinforcement Learning in Agent-Based Market Simulation: Unveiling Realistic Stylized Facts and Behavior](https://mlsanity.com/paper/2403.19781/) (2024)
- [Modelling Opaque Bilateral Market Dynamics in Financial Trading: Insights from a Multi-Agent Simulation Study](https://mlsanity.com/paper/2405.02849/) (2024)
- [Decoding OTC Government Bond Market Liquidity: An ABM Model for Market Dynamics](https://mlsanity.com/paper/2501.16331/) (2024)
- [Reinforcement Learning in High-frequency Market Making](https://mlsanity.com/paper/2407.21025/) (2024)
- [Natural-gas storage modelling by deep reinforcement learning](https://mlsanity.com/paper/2511.02646/) (2025)

## Abstract (arXiv)

We present a new financial framework where two families of RL-based agents representing the Liquidity Providers and Liquidity Takers learn simultaneously to satisfy their objective. Thanks to a parametrized reward formulation and the use of Deep RL, each group learns a shared policy able to generalize and interpolate over a wide range of behaviors. This is a step towards a fully RL-based market simulator replicating complex market conditions particularly suited to study the dynamics of the financial market under various scenarios.