mlsanityData to 14 Oct 2026

Optimal Execution with Reinforcement Learning

arXiv 2411.06389 ↗First posted 10 Nov 2024q-fin.TR cs.LGcited by 5

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

This study investigates the development of an optimal execution strategy through reinforcement learning, aiming to determine the most effective approach for traders to buy and sell inventory within a finite time horizon. Our proposed model leverages input features derived from the current state of the limit order book and operates at a high frequency to maximize control. To simulate this environment and overcome the limitations associated with relying on historical data, we utilize the multi-agent market simulator ABIDES, which provides a diverse range of depth levels within the limit order book. We present a custom MDP formulation followed by the results of our methodology and benchmark the performance against standard execution strategies. Results show that the reinforcement learning agent outperforms standard strategies and offers a practical foundation for real-world trading applications.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 11 of its 58 references that are in the map. First posted on arXiv 10 Nov 2024.

Builds on

11 of 58 references.

References · 11

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  4. 2018Double Deep Q-Learning for Optimal Executioncited by 43
  5. 2021ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Marketscited by 36
  6. 2020Generating Realistic Stock Market Order Streamscited by 30
  7. 2022Learning to simulate realistic limit order book markets from data as a World Agentcited by 27
  8. 2019How to Evaluate Trading Strategies: Single Agent Market Replay or Multiple Agent Interactive Simulation?cited by 15
  9. 2023Neural Stochastic Agent-Based Limit Order Book Simulation: A Hybrid Methodologycited by 11
  10. 2023Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Marketscited by 3
  11. 2022A Modular Framework for Reinforcement Learning Optimal Executioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2026Agentic Trading: When LLM Agents Meet Financial Marketscited by 7
  2. 2025Reinforcement Learning in Queue-Reactive Models: Application to Optimal Executioncited by 2
  3. 2025FlowOE: Imitation Learning with Flow Policy from Ensemble RL Experts for Optimal Execution under Heston Volatility and Concave Market Impactscited by 1
  4. 2026Diverse Approaches to Optimal Execution Schedule Generationcited by 0
  5. 2026Deep-MKV-TS: Path-Dependent McKean--Vlasov Control for Financial Time Series Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Right Place, Right Time: Market Simulation-based RL for Execution Optimisation
  2. 2025Reinforcement Learning for Stock Transactions
  3. 2022Imitate then Transcend: Multi-Agent Optimal Execution with Dual-Window Denoise PPO
  4. 2025Reinforcement Learning-Based Market Making as a Stochastic Control on Non-Stationary Limit Order Book Dynamics
  5. 2024Deep Reinforcement Learning for Online Optimal Execution Strategies
  6. 2025ABIDES-MARL: A Multi-Agent Reinforcement Learning Environment for Optimal Execution with Endogenous Liquidity ↗
  7. 2023Towards Generalizable Reinforcement Learning for Trade Execution
  8. 2026When a Correct Reward Is Not Enough: Diagnosing and Guiding PPO in an Analytically Solved Broker-Trader Game