# Optimal Market Making by Reinforcement Learning

arXiv: https://arxiv.org/abs/2104.04036 · HTML: https://mlsanity.com/paper/2104.04036/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 10 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Builds on (most cited first, 5 of 5)

- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [Risk-sensitive Reinforcement Learning](https://mlsanity.com/paper/1311.2097/) (2013)
- [Market Making via Reinforcement Learning](https://mlsanity.com/paper/1804.04216/) (2018)
- [Reinforcement Learning for Portfolio Management](https://mlsanity.com/paper/1909.09571/) (2019)

## Built on by (most cited first, 4 of 4)

- [A Review of Reinforcement Learning in Financial Applications](https://mlsanity.com/paper/2411.12746/) (2024)
- [Predicting Liquidity-Aware Bond Yields using Causal GANs and Deep Reinforcement Learning with LLM Evaluation](https://mlsanity.com/paper/2502.17011/) (2025)
- [Predictive Crypto-Asset Automated Market Making Architecture for Decentralized Finance using Deep Reinforcement Learning](https://mlsanity.com/paper/2211.01346/) (2022)
- [Reinforcement Learning-Based Market Making as a Stochastic Control on Non-Stationary Limit Order Book Dynamics](https://mlsanity.com/paper/2509.12456/) (2025)

## Abstract (arXiv)

We apply Reinforcement Learning algorithms to solve the classic quantitative finance Market Making problem, in which an agent provides liquidity to the market by placing buy and sell orders while maximizing a utility function. The optimal agent has to find a delicate balance between the price risk of her inventory and the profits obtained by capturing the bid-ask spread. We design an environment with a reward function that determines an order relation between policies equivalent to the original utility function. When comparing our agents with the optimal solution and a benchmark symmetric agent, we find that the Deep Q-Learning algorithm manages to recover the optimal agent.