# Market Making via Reinforcement Learning

arXiv: https://arxiv.org/abs/1804.04216 · HTML: https://mlsanity.com/paper/1804.04216/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2018.

## Built on by (most cited first, 30 of 47)

- [Recent Advances in Reinforcement Learning in Finance](https://mlsanity.com/paper/2112.04553/) (2021)
- [Towards Realistic Market Simulations: a Generative Adversarial Networks Approach](https://mlsanity.com/paper/2110.13287/) (2021)
- [Machine learning and deep learning](https://mlsanity.com/paper/2104.05314/) (2021)
- [Reinforcement Learning for Quantitative Trading](https://mlsanity.com/paper/2109.13851/) (2021)
- [Bayesian Bellman Operators](https://mlsanity.com/paper/2106.05012/) (2021)
- [Conditional Generators for Limit Order Book Environments: Explainability, Challenges, and Robustness](https://mlsanity.com/paper/2306.12806/) (2023)
- [JAX-LOB: A GPU-Accelerated limit order book simulator to unlock large scale reinforcement learning for trading](https://mlsanity.com/paper/2308.13289/) (2023)
- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulations](https://mlsanity.com/paper/2210.07184/) (2022)
- [Quantum Machine Learning for Finance](https://mlsanity.com/paper/2109.04298/) (2021)
- [Reinforcement Learning with Maskable Stock Representation for Portfolio Management in Customizable Stock Pools](https://mlsanity.com/paper/2311.10801/) (2023)
- [IMM: An Imitative Reinforcement Learning Approach with Predictive Representation Learning for Automatic Market Making](https://mlsanity.com/paper/2308.08918/) (2023)
- [Market Making with Scaled Beta Policies](https://mlsanity.com/paper/2207.03352/) (2022)
- [PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets](https://mlsanity.com/paper/2302.00586/) (2023)
- [Uniswap Liquidity Provision: An Online Learning Approach](https://mlsanity.com/paper/2302.00610/) (2023)
- [LOB-Bench: Benchmarking Generative AI for Finance -- an Application to Limit Order Book Data](https://mlsanity.com/paper/2502.09172/) (2025)
- [Optimal Market Making by Reinforcement Learning](https://mlsanity.com/paper/2104.04036/) (2021)
- [FORM: Learning Expressive and Transferable First-Order Logic Reward Machines](https://mlsanity.com/paper/2501.00364/) (2024)
- [Computational Performance of Deep Reinforcement Learning to find Nash Equilibria](https://mlsanity.com/paper/2104.12895/) (2021)
- [Robust Risk-Sensitive Reinforcement Learning Agents for Trading Markets](https://mlsanity.com/paper/2107.08083/) (2021)
- [Model-based gym environments for limit order book trading](https://mlsanity.com/paper/2209.07823/) (2022)
- [Deep Reinforcement Learning for Market Making Under a Hawkes Process-Based Limit Order Book Model](https://mlsanity.com/paper/2207.09951/) (2022)
- [Limited or Biased: Modeling Sub-Rational Human Investors in Financial Markets](https://mlsanity.com/paper/2210.08569/) (2022)
- [Neural Augmented Kalman Filtering with Bollinger Bands for Pairs Trading](https://mlsanity.com/paper/2210.15448/) (2022)
- [Reinforcement Learning Applied to Trading Systems: A Survey](https://mlsanity.com/paper/2212.06064/) (2022)
- [Robust Market Making: To Quote, or not To Quote](https://mlsanity.com/paper/2508.16588/) (2025)
- [ARL-Based Multi-Action Market Making with Hawkes Processes and Variable Volatility](https://mlsanity.com/paper/2508.16589/) (2025)
- [Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market](https://mlsanity.com/paper/2112.04494/) (2021)
- [Predictive Crypto-Asset Automated Market Making Architecture for Decentralized Finance using Deep Reinforcement Learning](https://mlsanity.com/paper/2211.01346/) (2022)
- [Market Making with Deep Reinforcement Learning from Limit Order Books](https://mlsanity.com/paper/2305.15821/) (2023)

## Abstract (arXiv)

Market making is a fundamental trading problem in which an agent provides liquidity by continually offering to buy and sell a security. The problem is challenging due to inventory risk, the risk of accumulating an unfavourable position and ultimately losing money. In this paper, we develop a high-fidelity simulation of limit order book markets, and use it to design a market making agent using temporal-difference reinforcement learning. We use a linear combination of tile codings as a value function approximator, and design a custom reward function that controls inventory risk. We demonstrate the effectiveness of our approach by showing that our agent outperforms both simple benchmark strategies and a recent online learning approach from the literature.