mlsanityData to 14 Oct 2026

Reinforcement Learning with Hawkes-Structured Networks for Market Making

arXiv 2610.16524 ↗First posted 12 Oct 2026cs.LG q-fin.CPcited by 0

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

Market making under realistic order-flow dynamics is challenging due to the clustered, self-exciting nature of order arrivals, which are naturally modeled by non-Markovian Hawkes processes. We propose a model-free reinforcement learning (RL) framework for market making with general Hawkes order-flow dynamics. We first establish non-asymptotic performance bounds that quantify the loss incurred by approximating the infinite-dimensional Hawkes state using a finite collection of exponentially weighted features. Guided by this analysis, we introduce the Hawkes-Structured Network (HSN), a novel neural architecture that encodes historical order flow through a learnable mixture-of-exponentials filter. HSN is trained end-to-end using model-free RL, jointly optimizing the feature representation and quoting policy directly from reward feedback. Experiments on single- and multi-asset market-making problems with mixture-of-exponentials, power-law, and regime-switching kernels demonstrate that HSN outperforms general-purpose neural architectures, achieving higher trading rewards while implicitly recovering the underlying order-flow intensities.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 11 of its 41 references that are in the map. First posted on arXiv 12 Oct 2026.

Builds on

11 of 41 references.

References · 11

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  5. 2021Recent Advances in Reinforcement Learning in Financecited by 110
  6. 2018Market Making via Reinforcement Learningcited by 47
  7. 2019Reinforcement Learning for Market Making in a Multi-agent Dealer Marketcited by 39
  8. 2023Limit Order Book Dynamics and Order Size Modelling Using Compound Hawkes Processcited by 5
  9. 2022Deep Reinforcement Learning for Market Making Under a Hawkes Process-Based Limit Order Book Modelcited by 2
  10. 2024Logarithmic regret in the ergodic Avellaneda-Stoikov market making modelcited by 2
  11. 2021Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Marketcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Robust Market Making with Hawkes Order Flow and Price Impact via Adversarial Reinforcement Learning
  2. 2025ARL-Based Multi-Action Market Making with Hawkes Processes and Variable Volatility
  3. 2023IMM: An Imitative Reinforcement Learning Approach with Predictive Representation Learning for Automatic Market Making
  4. 2026ViperQ: Order Flow Pattern Recognition via Auction Market Theory for Reinforcement Learning Trading
  5. 2025Reinforcement Learning in Queue-Reactive Models: Application to Optimal Execution
  6. 2025Reinforcement Learning-Based Market Making as a Stochastic Control on Non-Stationary Limit Order Book Dynamics
  7. 2025Deep reinforcement learning for optimal trading with partial information
  8. 2026Reinforcement Learning for Execution under Dynamic Fees in a Closed-Loop DEX Simulator