# Optimizing Market Making using Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/1812.10252 · HTML: https://mlsanity.com/paper/1812.10252/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Dec 2018.

## Built on by (most cited first, 13 of 13)

- [Recent Advances in Reinforcement Learning in Finance](https://mlsanity.com/paper/2112.04553/) (2021)
- [A Review of Reinforcement Learning in Financial Applications](https://mlsanity.com/paper/2411.12746/) (2024)
- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [Factored Policy Gradients: Leveraging Structure for Efficient Learning in MOMDPs](https://mlsanity.com/paper/2102.10362/) (2021)
- [IMM: An Imitative Reinforcement Learning Approach with Predictive Representation Learning for Automatic Market Making](https://mlsanity.com/paper/2308.08918/) (2023)
- [Market Making with Scaled Beta Policies](https://mlsanity.com/paper/2207.03352/) (2022)
- [Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges](https://mlsanity.com/paper/2305.10091/) (2023)
- [Model-based gym environments for limit order book trading](https://mlsanity.com/paper/2209.07823/) (2022)
- [Multi-Agent Reinforcement Learning for Market Making: Competition without Collusion](https://mlsanity.com/paper/2510.25929/) (2025)
- [Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market](https://mlsanity.com/paper/2112.04494/) (2021)
- [Integrating Tick-level Data and Periodical Signal for High-frequency Market Making](https://mlsanity.com/paper/2306.17179/) (2023)
- [INTAGS: Interactive Agent-Guided Simulation](https://mlsanity.com/paper/2309.01784/) (2023)
- [Reinforcement Learning-Based Market Making as a Stochastic Control on Non-Stationary Limit Order Book Dynamics](https://mlsanity.com/paper/2509.12456/) (2025)

## Abstract (arXiv)

In this paper, reinforcement learning is applied to the problem of optimizing market making. A multi-agent reinforcement learning framework is used to optimally place limit orders that lead to successful trades. The framework consists of two agents. The macro-agent optimizes on making the decision to buy, sell, or hold an asset. The micro-agent optimizes on placing limit orders within the limit order book. For the context of this paper, the proposed framework is applied and studied on the Bitcoin cryptocurrency market. The goal of this paper is to show that reinforcement learning is a viable strategy that can be applied to complex problems (with complex environments) such as market making.