# Robust Market Making with Hawkes Order Flow and Price Impact via Adversarial Reinforcement Learning

arXiv: https://arxiv.org/abs/2609.22785 · HTML: https://mlsanity.com/paper/2609.22785/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 37 references that are in the map.
- First posted on arXiv 19 Sep 2026.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [A Distributional Perspective on Reinforcement Learning](https://mlsanity.com/paper/1707.06887/) (2017)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [EPOpt: Learning Robust Neural Network Policies Using Model Ensembles](https://mlsanity.com/paper/1610.01283/) (2016)
- [Market Making via Reinforcement Learning](https://mlsanity.com/paper/1804.04216/) (2018)
- [Robust Market Making via Adversarial Reinforcement Learning](https://mlsanity.com/paper/2003.01820/) (2020)
- [Risk-Sensitive Compact Decision Trees for Autonomous Execution in Presence of Simulated Market Response](https://mlsanity.com/paper/1906.02312/) (2019)
- [Model-based gym environments for limit order book trading](https://mlsanity.com/paper/2209.07823/) (2022)
- [Optimal market making with persistent order flow](https://mlsanity.com/paper/2003.05958/) (2020)
- [Robust Market Making: To Quote, or not To Quote](https://mlsanity.com/paper/2508.16588/) (2025)
- [ARL-Based Multi-Action Market Making with Hawkes Processes and Variable Volatility](https://mlsanity.com/paper/2508.16589/) (2025)
- [A Model of Market Making and Price Impact](https://mlsanity.com/paper/2101.01388/) (2021)
- [Market Making with Deep Reinforcement Learning from Limit Order Books](https://mlsanity.com/paper/2305.15821/) (2023)

## Abstract (arXiv)

Market-making strategies in real limit order book markets face substantial model uncertainty and regime-shift risk. Existing adversarial reinforcement learning approaches improve robustness by formulating the Avellaneda--Stoikov market-making problem as a zero-sum game between a market maker and an environmental adversary. However, these approaches typically rely on Poisson order arrivals and neglect trade-induced price impact, limiting their ability to capture important high-frequency market microstructure effects such as clustered order flow, self-excitation, and post-trade price feedback. We extend adversarial reinforcement learning for market making to a more complex environment with Hawkes self-exciting order arrivals and trade-induced price impact. To mitigate the increased non-stationarity introduced by the expanded regime space, we incorporate an LSTM module that explicitly models the temporal structure of recent observations. We further characterize the equilibrium properties of the proposed framework through both game-theoretic analysis and numerical experiments, and introduce a robustness evaluation protocol focused on improvements in the left tail of the return distribution. Experimental results across a range of market regimes show that the proposed method achieves improved left-tail performance in most complex microstructure environments. In particular, the gains are pronounced in regimes with strong Hawkes excitation and low-to-moderate price impact. Bootstrap tests provide no evidence that these improvements are obtained through a stronger terminal directional inventory bias. These results suggest that combining adversarial training with temporal state representation can improve the robustness of reinforcement-learning-based market-making strategies under order-flow self-excitation, price impact, and regime uncertainty.