# Market Making with Deep Reinforcement Learning from Limit Order Books

arXiv: https://arxiv.org/abs/2305.15821 · HTML: https://mlsanity.com/paper/2305.15821/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 53 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 15 of 15)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Universal features of price formation in financial markets: perspectives from Deep Learning](https://mlsanity.com/paper/1803.06917/) (2018)
- [Benchmark Dataset for Mid-Price Forecasting of Limit Order Book Data with Machine Learning Methods](https://mlsanity.com/paper/1705.03233/) (2017)
- [Market Making via Reinforcement Learning](https://mlsanity.com/paper/1804.04216/) (2018)
- [Temporal Attention augmented Bilinear Network for Financial Time-Series Data Analysis](https://mlsanity.com/paper/1712.00975/) (2017)
- [Using Deep Learning for price prediction by exploiting stationary limit order book features](https://mlsanity.com/paper/1810.09965/) (2018)
- [Deep Reinforcement Learning for Active High Frequency Trading](https://mlsanity.com/paper/2101.07107/) (2021)
- [Deep Stock Trading: A Hierarchical Reinforcement Learning Framework for Portfolio Optimization and Order Execution](https://mlsanity.com/paper/2012.12620/) (2020)
- [Machine Learning for Forecasting Mid Price Movement using Limit Order Book Data](https://mlsanity.com/paper/1809.07861/) (2018)
- [Model-based Reinforcement Learning for Predictions and Control for Limit Order Books](https://mlsanity.com/paper/1910.03743/) (2019)
- [Multi-Horizon Forecasting for Limit Order Books: Novel Deep Learning Approaches and Hardware Acceleration using Intelligent Processing Units](https://mlsanity.com/paper/2105.10430/) (2021)
- [Tensor Representation in High-Frequency Financial Data for Price Change Prediction](https://mlsanity.com/paper/1709.01268/) (2017)
- [Multi-head Temporal Attention-Augmented Bilinear Network for Financial time series prediction](https://mlsanity.com/paper/2201.05459/) (2022)
- [Feature Engineering for Mid-Price Prediction with Deep Learning](https://mlsanity.com/paper/1904.05384/) (2019)

## Built on by (most cited first, 1 of 1)

- [Robust Market Making with Hawkes Order Flow and Price Impact via Adversarial Reinforcement Learning](https://mlsanity.com/paper/2609.22785/) (2026)

## Abstract (arXiv)

Market making (MM) is an important research topic in quantitative finance, the agent needs to continuously optimize ask and bid quotes to provide liquidity and make profits. The limit order book (LOB) contains information on all active limit orders, which is an essential basis for decision-making. The modeling of evolving, high-dimensional and low signal-to-noise ratio LOB data is a critical challenge. Traditional MM strategy relied on strong assumptions such as price process, order arrival process, etc. Previous reinforcement learning (RL) works handcrafted market features, which is insufficient to represent the market. This paper proposes a RL agent for market making with LOB data. We leverage a neural network with convolutional filters and attention mechanism (Attn-LOB) for feature extraction from LOB. We design a new continuous action space and a hybrid reward function for the MM task. Finally, we conduct comprehensive experiments on latency and interpretability, showing that our agent has good applicability.