mlsanityData to 14 Oct 2026

Double Deep Q-Learning for Optimal Execution

arXiv 1812.06600 ↗First posted 17 Dec 2018q-fin.TR cs.LG q-fin.CPcited by 43

Not on the map. Why?

Abstract (arXiv)

Optimal trade execution is an important problem faced by essentially all traders. Much research into optimal execution uses stringent model assumptions and applies continuous time stochastic control to solve them. Here, we instead take a model free approach and develop a variation of Deep Q-Learning to estimate the optimal actions of a trader. The model is a fully connected Neural Network trained using Experience Replay and Double DQN with input features given by the current state of the limit order book, other trading signals, and available execution actions, while the output is the Q-value function estimating the future rewards under an arbitrary action. We apply our model to nine different stocks and find that it outperforms the standard benchmark approach on most stocks using the measures of (i) mean and median out-performance, (ii) probability of out-performance, and (iii) gain-loss ratios.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 5; 2023: 7; 2024: 8; 2025: 6; 2026: 13.

43 papers.

Sort:

By area:

  1. 2021Recent Advances in Reinforcement Learning in Financecited by 110
  2. 2021Universal Trading for Order Execution with Oracle Policy Distillationcited by 30
  3. 2021Reinforcement Learning for Quantitative Tradingcited by 24
  4. 2023Conditional Generators for Limit Order Book Environments: Explainability, Challenges, and Robustnesscited by 14
  5. 2023JAX-LOB: A GPU-Accelerated limit order book simulator to unlock large scale reinforcement learning for tradingcited by 14
  6. 2024The Evolution of Reinforcement Learning in Quantitative Finance: A Surveycited by 11
  7. 2025From Deep Learning to LLMs: A survey of AI in Quantitative Investmentcited by 9
  8. 2023Learning Multi-Agent Intention-Aware Communication for Optimal Multi-Order Execution in Financecited by 7
  9. 2023Towards Generalizable Reinforcement Learning for Trade Executioncited by 5
  10. 2024Optimal Execution with Reinforcement Learningcited by 5
  11. 2022Learn Continuously, Act Discretely: Hybrid Action-Space Reinforcement Learning For Optimal Executioncited by 4
  12. 2024Deep Hedging with Market Impactcited by 4
31 more
  1. 2021Robust Risk-Sensitive Reinforcement Learning Agents for Trading Marketscited by 3
  2. 2023Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Marketscited by 3
  3. 2022Applications of Reinforcement Learning in Finance -- Trading with a Double Deep Q-Networkcited by 2
  4. 2024Q-Learning under Finite Model Uncertaintycited by 2
  5. 2024Deep Reinforcement Learning for Online Optimal Execution Strategiescited by 2
  6. 2025Safe and Compliant Cross-Market Trade Execution via Constrained RL and Zero-Knowledge Auditscited by 2
  7. 2025Reinforcement Learning in Queue-Reactive Models: Application to Optimal Executioncited by 2
  8. 2026Learning in Markov Decision Processes with Exogenous Dynamicscited by 2
  9. 2026Memory-Induced Supra-Competitive Outcomes Between Deep Reinforcement Learning Agents in Optimal Trade Executioncited by 2
  10. 2022Imitate then Transcend: Multi-Agent Optimal Execution with Dual-Window Denoise PPOcited by 1
  11. 2022A Modular Framework for Reinforcement Learning Optimal Executioncited by 1
  12. 2022Hierarchical Deep Reinforcement Learning for VWAP Strategy Optimizationcited by 1
  13. 2023Optimal Execution Using Reinforcement Learningcited by 1
  14. 2024Deviations from the Nash equilibrium in a two-player optimal execution game with reinforcement learningcited by 1
  15. 2024Trading through Earnings Seasons using Self-Supervised Contrastive Representation Learningcited by 1
  16. 2025Deep reinforcement learning for optimal trading with partial informationcited by 1
  17. 2026Learning Market Making with Closing Auctionscited by 1
  18. 2023Reinforcement Learning for Financial Index Trackingcited by 0
  19. 2024Two Kinds of Learning Algorithms for Continuous-Time VWAP Targeting Executioncited by 0
  20. 2025Risk-averse policies for natural gas futures trading using distributional reinforcement learningcited by 0
  21. 2025The Role of Deep Learning in Financial Asset Management: A Systematic Reviewcited by 0
  22. 2026RAmmStein: Regime Adaptation in Mean-reverting Markets with Stein Thresholds -- Optimal Impulse Control in Concentrated AMMscited by 0
  23. 2026When Quotes Crumble: Detecting Transient Mechanical Liquidity Erosion in Limit Order Bookscited by 0
  24. 2026Can Reinforcement Learning Efficiently Discover Price Manipulation?cited by 0
  25. 2026Reinforcement Learning for Execution under Dynamic Fees in a Closed-Loop DEX Simulatorcited by 0
  26. 2026NeuralChaos: Optimal Adapted Approximation of Square Integrable Predictable Processescited by 0
  27. 2026Can Large Language Models Execute Parent Orders?cited by 0
  28. 2026FARE: Deep Reinforcement Learning For Fair Exposure Constrained Uncertainty Aware Financial Content Personalizationcited by 0
  29. 2026Beyond Supra-Competitive Outcomes: Collusive Behaviour in Deep Reinforcement Learning for Optimal Execution Gamescited by 0
  30. 2026Mixture-of-Experts for Cryptocurrency Order Execution: Training Stability, Tail Risk, and Failure Modescited by 0
  31. 2026When a Correct Reward Is Not Enough: Diagnosing and Guiding PPO in an Analytically Solved Broker-Trader Gamecited by 0