# Deep Reinforcement Learning for Active High Frequency Trading

arXiv: https://arxiv.org/abs/2101.07107 · HTML: https://mlsanity.com/paper/2101.07107/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 57 references that are in the map.
- First posted on arXiv 18 Jan 2021.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [Risk-sensitive Reinforcement Learning](https://mlsanity.com/paper/1311.2097/) (2013)
- [Deep Reinforcement Learning for Trading](https://mlsanity.com/paper/1911.10107/) (2019)
- [Universal features of price formation in financial markets: perspectives from Deep Learning](https://mlsanity.com/paper/1803.06917/) (2018)
- [Cryptocurrency Portfolio Management with Deep Reinforcement Learning](https://mlsanity.com/paper/1612.01277/) (2016)
- [ABIDES: Towards High-Fidelity Market Simulation for AI Research](https://mlsanity.com/paper/1904.12066/) (2019)
- [An Application of Deep Reinforcement Learning to Algorithmic Trading](https://mlsanity.com/paper/2004.06627/) (2020)
- [How to Evaluate Trading Strategies: Single Agent Market Replay or Multiple Agent Interactive Simulation?](https://mlsanity.com/paper/1906.12010/) (2019)
- [Deep Learning modeling of Limit Order Book: a comparative perspective](https://mlsanity.com/paper/2007.07319/) (2020)
- [Deep Learning for Portfolio Optimization](https://mlsanity.com/paper/2005.13665/) (2020)

## Built on by (most cited first, 21 of 21)

- [Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking](https://mlsanity.com/paper/2505.11065/) (2025)
- [LiveTradeBench: Seeking Real-World Alpha with Large Language Models](https://mlsanity.com/paper/2511.03628/) (2025)
- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [Multi-Horizon Forecasting for Limit Order Books: Novel Deep Learning Approaches and Hardware Acceleration using Intelligent Processing Units](https://mlsanity.com/paper/2105.10430/) (2021)
- [Deep Learning for Market by Order Data](https://mlsanity.com/paper/2102.08811/) (2021)
- [HLOB -- Information Persistence and Structure in Limit Order Books](https://mlsanity.com/paper/2405.18938/) (2024)
- [Deep Limit Order Book Forecasting](https://mlsanity.com/paper/2403.09267/) (2024)
- [MacroHFT: Memory Augmented Context-aware Reinforcement Learning On High Frequency Trading](https://mlsanity.com/paper/2406.14537/) (2024)
- [Sparsification and Filtering for Spatial-temporal GNN in Multivariate Time-series](https://mlsanity.com/paper/2203.03991/) (2022)
- [Will LLMs be Professional at Fund Investment? DeepFund: A Live Arena Perspective](https://mlsanity.com/paper/2503.18313/) (2025)
- [Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Markets](https://mlsanity.com/paper/2301.08688/) (2023)
- [FlowHFT: Imitation Learning via Flow Matching Policy for Optimal High-Frequency Trading under Diverse Market Conditions](https://mlsanity.com/paper/2505.05784/) (2025)
- [Market Making with Deep Reinforcement Learning from Limit Order Books](https://mlsanity.com/paper/2305.15821/) (2023)
- ["So, Tell Me About Your Policy...": Distillation of interpretable policies from Deep Reinforcement Learning agents](https://mlsanity.com/paper/2507.07848/) (2025)
- [Deep reinforcement learning for optimal trading with partial information](https://mlsanity.com/paper/2511.00190/) (2025)
- [FineFT: Efficient and Risk-Aware Ensemble Reinforcement Learning for Futures Trading](https://mlsanity.com/paper/2512.23773/) (2025)
- [Trading Devil RL: Backdoor attack via Stock market, Bayesian Optimization and Reinforcement Learning](https://mlsanity.com/paper/2412.17908/) (2024)
- [Your Offline Policy is Not Trustworthy: Bilevel Reinforcement Learning for Sequential Portfolio Optimization](https://mlsanity.com/paper/2505.12759/) (2025)
- [Deep Reinforcement Learning in Factor Investment](https://mlsanity.com/paper/2509.16206/) (2025)
- [ViperQ: Order Flow Pattern Recognition via Auction Market Theory for Reinforcement Learning Trading](https://mlsanity.com/paper/2609.13825/) (2026)
- [SOTA: Stock Options Trading Agents Guided by Option-Implied Return Distributions](https://mlsanity.com/paper/2610.10407/) (2026)

## Abstract (arXiv)

We introduce the first end-to-end Deep Reinforcement Learning (DRL) based framework for active high frequency trading in the stock market. We train DRL agents to trade one unit of Intel Corporation stock by employing the Proximal Policy Optimization algorithm. The training is performed on three contiguous months of high frequency Limit Order Book data, of which the last month constitutes the validation data. In order to maximise the signal to noise ratio in the training data, we compose the latter by only selecting training samples with largest price changes. The test is then carried out on the following month of data. Hyperparameters are tuned using the Sequential Model Based Optimization technique. We consider three different state characterizations, which differ in their LOB-based meta-features. Analysing the agents' performances on test data, we argue that the agents are able to create a dynamic representation of the underlying environment. They identify occasional regularities present in the data and exploit them to create long-term profitable trading strategies. Indeed, agents learn trading strategies able to produce stable positive returns in spite of the highly stochastic and non-stationary environment.