mlsanityData to 14 Oct 2026

Reinforcement Learning in High-frequency Market Making

arXiv 2407.21025 ↗First posted 14 Jul 2024q-fin.TR cs.LG econ.EMcited by 0

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

This paper establishes a new and comprehensive theoretical analysis for the application of reinforcement learning (RL) in high-frequency market making. We bridge the modern RL theory and the continuous-time statistical models in high-frequency financial economics. Different with most existing literature on methodological research about developing various RL methods for market making problem, our work is a pilot to provide the theoretical analysis. We target the effects of sampling frequency, and find an interesting tradeoff between error and complexity of RL algorithm when tweaking the values of the time increment Delta - as Delta becomes smaller, the error will be smaller but the complexity will be larger. We also study the two-player case under the general-sum game framework and establish the convergence of Nash equilibrium to the continuous-time game equilibrium as Deltarightarrow0. The Nash Q-learning algorithm, which is an online multi-agent RL method, is applied to solve the equilibrium. Our theories are not only useful for practitioners to choose the sampling frequency, but also very general and applicable to other high-frequency financial decision making problems, e.g., optimal executions, as long as the time-discretization of a continuous-time markov decision process is adopted. Monte Carlo simulation evidence support all of our theories.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 7 of its 41 references that are in the map. First posted on arXiv 14 Jul 2024.

Builds on

7 of 41 references.

References · 7

  1. 2021Recent Advances in Reinforcement Learning in Financecited by 110
  2. 2019Reinforcement Learning for Market Making in a Multi-agent Dealer Marketcited by 39
  3. 2018Q-learning with Nearest Neighborscited by 37
  4. 2021Towards a fully RL-based Market Simulatorcited by 10
  5. 2022Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulationscited by 10
  6. 2022Approximate Q-Learning for Controlled Diffusion Processes and its Near Optimalitycited by 7
  7. 2022Can maker-taker fees prevent algorithmic cooperation in market making?cited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A non-zero-sum game with reinforcement learning under mean-variance framework
  2. 2024The Evolution of Reinforcement Learning in Quantitative Finance: A Survey
  3. 2024Deviations from the Nash equilibrium in a two-player optimal execution game with reinforcement learning
  4. 2025Reinforcement Learning-Based Market Making as a Stochastic Control on Non-Stationary Limit Order Book Dynamics
  5. 2025From Individual Learning to Market Equilibrium: Correcting Structural and Parametric Biases in RL Simulations of Economic Models
  6. 2026A Survey of Reinforcement Learning For Economics
  7. 2025When AI Trading Agents Compete: Adverse Selection of Meta-Orders by Reinforcement Learning-Based Market Making
  8. 2024Reinforcement Learning in Agent-Based Market Simulation: Unveiling Realistic Stylized Facts and Behavior