# Robust Market Making via Adversarial Reinforcement Learning

arXiv: https://arxiv.org/abs/2003.01820 · HTML: https://mlsanity.com/paper/2003.01820/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2020.

## Built on by (most cited first, 14 of 14)

- [Automate Strategy Finding with LLM in Quant Investment](https://mlsanity.com/paper/2409.06289/) (2024)
- [Reinforcement Learning for Quantitative Trading](https://mlsanity.com/paper/2109.13851/) (2021)
- [A Review of Reinforcement Learning in Financial Applications](https://mlsanity.com/paper/2411.12746/) (2024)
- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [Market Making with Scaled Beta Policies](https://mlsanity.com/paper/2207.03352/) (2022)
- [Model-based gym environments for limit order book trading](https://mlsanity.com/paper/2209.07823/) (2022)
- [STORM: A Spatio-Temporal Factor Model Based on Dual Vector Quantized Variational Autoencoders for Financial Trading](https://mlsanity.com/paper/2412.09468/) (2024)
- [ARL-Based Multi-Action Market Making with Hawkes Processes and Variable Volatility](https://mlsanity.com/paper/2508.16589/) (2025)
- [Multi-Agent Reinforcement Learning for Market Making: Competition without Collusion](https://mlsanity.com/paper/2510.25929/) (2025)
- [Reward is not enough: can we liberate AI from the reinforcement learning paradigm?](https://mlsanity.com/paper/2202.03192/) (2022)
- [Solving The Dynamic Volatility Fitting Problem: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2410.11789/) (2024)
- [The Invisible Handshake: Persistent Overpricing by Adaptive Market Agents](https://mlsanity.com/paper/2510.15995/) (2025)
- [Robust Market Making with Hawkes Order Flow and Price Impact via Adversarial Reinforcement Learning](https://mlsanity.com/paper/2609.22785/) (2026)
- [When and How Order-Flow Interventions Work After Dealers Adapt](https://mlsanity.com/paper/2610.18158/) (2026)

## Abstract (arXiv)

We show that adversarial reinforcement learning (ARL) can be used to produce market marking agents that are robust to adversarial and adaptively-chosen market conditions. To apply ARL, we turn the well-studied single-agent model of Avellaneda and Stoikov [2008] into a discrete-time zero-sum game between a market maker and adversary. The adversary acts as a proxy for other market participants that would like to profit at the market maker's expense. We empirically compare two conventional single-agent RL agents with ARL, and show that our ARL approach leads to: 1) the emergence of risk-averse behaviour without constraints or domain-specific penalties; 2) significant improvements in performance across a set of standard metrics, evaluated with or without an adversary in the test environment, and; 3) improved robustness to model uncertainty. We empirically demonstrate that our ARL method consistently converges, and we prove for several special cases that the profiles that we converge to correspond to Nash equilibria in a simplified single-stage game.