# Combining Deep Learning on Order Books with Reinforcement Learning for Profitable Trading

arXiv: https://arxiv.org/abs/2311.02088 · HTML: https://mlsanity.com/paper/2311.02088/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 33 references that are in the map.
- First posted on arXiv 24 Oct 2023.

## Builds on (most cited first, 4 of 4)

- [Market Making via Reinforcement Learning](https://mlsanity.com/paper/1804.04216/) (2018)
- [Temporal Attention augmented Bilinear Network for Financial Time-Series Data Analysis](https://mlsanity.com/paper/1712.00975/) (2017)
- [Temporal Logistic Neural Bag-of-Features for Financial Time series Forecasting leveraging Limit Order Book Data](https://mlsanity.com/paper/1901.08280/) (2019)
- [Risk-Sensitive Compact Decision Trees for Autonomous Execution in Presence of Simulated Market Response](https://mlsanity.com/paper/1906.02312/) (2019)

## Built on by (most cited first, 1 of 1)

- [DeepSeekMath Meets Order Book: Group-Aware Policy Optimization for High-Frequency Directional Trading](https://mlsanity.com/paper/2605.25527/) (2026)

## Abstract (arXiv)

High-frequency trading is prevalent, where automated decisions must be made quickly to take advantage of price imbalances and patterns in price action that forecast near-future movements. While many algorithms have been explored and tested, analytical methods fail to harness the whole nature of the market environment by focusing on a limited domain. With the evergrowing machine learning field, many large-scale end-to-end studies on raw data have been successfully employed to increase the domain scope for profitable trading but are very difficult to replicate. Combining deep learning on the order books with reinforcement learning is one way of breaking down large-scale end-to-end learning into more manageable and lightweight components for reproducibility, suitable for retail trading. The following work focuses on forecasting returns across multiple horizons using order flow imbalance and training three temporal-difference learning models for five financial instruments to provide trading signals. The instruments used are two foreign exchange pairs (GBPUSD and EURUSD), two indices (DE40 and FTSE100), and one commodity (XAUUSD). The performances of these 15 agents are evaluated through backtesting simulation, and successful models proceed through to forward testing on a retail trading platform. The results prove potential but require further minimal modifications for consistently profitable trading to fully handle retail trading costs, slippage, and spread fluctuation.