# Optimal Execution with Reinforcement Learning

arXiv: https://arxiv.org/abs/2411.06389 · HTML: https://mlsanity.com/paper/2411.06389/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 58 references that are in the map.
- First posted on arXiv 10 Nov 2024.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 11 of 11)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Double Deep Q-Learning for Optimal Execution](https://mlsanity.com/paper/1812.06600/) (2018)
- [ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Markets](https://mlsanity.com/paper/2110.14771/) (2021)
- [Generating Realistic Stock Market Order Streams](https://mlsanity.com/paper/2006.04212/) (2020)
- [Learning to simulate realistic limit order book markets from data as a World Agent](https://mlsanity.com/paper/2210.09897/) (2022)
- [How to Evaluate Trading Strategies: Single Agent Market Replay or Multiple Agent Interactive Simulation?](https://mlsanity.com/paper/1906.12010/) (2019)
- [Neural Stochastic Agent-Based Limit Order Book Simulation: A Hybrid Methodology](https://mlsanity.com/paper/2303.00080/) (2023)
- [Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Markets](https://mlsanity.com/paper/2301.08688/) (2023)
- [A Modular Framework for Reinforcement Learning Optimal Execution](https://mlsanity.com/paper/2208.06244/) (2022)

## Built on by (most cited first, 5 of 5)

- [Agentic Trading: When LLM Agents Meet Financial Markets](https://mlsanity.com/paper/2605.19337/) (2026)
- [Reinforcement Learning in Queue-Reactive Models: Application to Optimal Execution](https://mlsanity.com/paper/2511.15262/) (2025)
- [FlowOE: Imitation Learning with Flow Policy from Ensemble RL Experts for Optimal Execution under Heston Volatility and Concave Market Impacts](https://mlsanity.com/paper/2506.05755/) (2025)
- [Diverse Approaches to Optimal Execution Schedule Generation](https://mlsanity.com/paper/2601.22113/) (2026)
- [Deep-MKV-TS: Path-Dependent McKean--Vlasov Control for Financial Time Series Generation](https://mlsanity.com/paper/2608.19394/) (2026)

## Abstract (arXiv)

This study investigates the development of an optimal execution strategy through reinforcement learning, aiming to determine the most effective approach for traders to buy and sell inventory within a finite time horizon. Our proposed model leverages input features derived from the current state of the limit order book and operates at a high frequency to maximize control. To simulate this environment and overcome the limitations associated with relying on historical data, we utilize the multi-agent market simulator ABIDES, which provides a diverse range of depth levels within the limit order book. We present a custom MDP formulation followed by the results of our methodology and benchmark the performance against standard execution strategies. Results show that the reinforcement learning agent outperforms standard strategies and offers a practical foundation for real-world trading applications.