# Financial Trading as a Game: A Deep Reinforcement Learning Approach

arXiv: https://arxiv.org/abs/1807.02787 · HTML: https://mlsanity.com/paper/1807.02787/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jul 2018.

## Built on by (most cited first, 22 of 22)

- [FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance](https://mlsanity.com/paper/2111.09395/) (2021)
- [Deep Reinforcement Learning in Quantitative Algorithmic Trading: A Review](https://mlsanity.com/paper/2106.00123/) (2021)
- [A Novel Deep Reinforcement Learning Based Automated Stock Trading System Using Cascaded LSTM Networks](https://mlsanity.com/paper/2212.02721/) (2022)
- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [Sales Time Series Analytics Using Deep Q-Learning](https://mlsanity.com/paper/2201.02058/) (2022)
- [What Teaches Robots to Walk, Teaches Them to Trade too -- Regime Adaptive Execution using Informed Data and LLMs](https://mlsanity.com/paper/2406.15508/) (2024)
- [High-Dimensional Stock Portfolio Trading with Deep Reinforcement Learning](https://mlsanity.com/paper/2112.04755/) (2021)
- [Applications of Reinforcement Learning in Finance -- Trading with a Double Deep Q-Network](https://mlsanity.com/paper/2206.14267/) (2022)
- [Predicting Liquidity-Aware Bond Yields using Causal GANs and Deep Reinforcement Learning with LLM Evaluation](https://mlsanity.com/paper/2502.17011/) (2025)
- [Analytics of Business Time Series Using Machine Learning and Bayesian Inference](https://mlsanity.com/paper/2205.12905/) (2022)
- [Deep Policy Gradient Methods in Commodity Markets](https://mlsanity.com/paper/2308.01910/) (2023)
- [NIFTY Financial News Headlines Dataset](https://mlsanity.com/paper/2405.09747/) (2024)
- [Reinforcement Learning Pair Trading: A Dynamic Scaling approach](https://mlsanity.com/paper/2407.16103/) (2024)
- [Trading through Earnings Seasons using Self-Supervised Contrastive Representation Learning](https://mlsanity.com/paper/2409.17392/) (2024)
- [Provably Efficient Action-Manipulation Attack Against Continuous Reinforcement Learning](https://mlsanity.com/paper/2411.13116/) (2024)
- [Deep Reinforcement Learning with Function Properties in Mean Reversion Strategies](https://mlsanity.com/paper/2101.03418/) (2021)
- [Similarity metrics for Different Market Scenarios in Abides](https://mlsanity.com/paper/2107.09352/) (2021)
- [An intelligent algorithmic trading based on a risk-return reinforcement learning algorithm](https://mlsanity.com/paper/2208.10707/) (2022)
- [An Ensemble Method of Deep Reinforcement Learning for Automated Cryptocurrency Trading](https://mlsanity.com/paper/2309.00626/) (2023)
- [Commodities Trading through Deep Policy Gradient Methods](https://mlsanity.com/paper/2309.00630/) (2023)
- [Exploiting Risk-Aversion and Size-dependent fees in FX Trading with Fitted Natural Actor-Critic](https://mlsanity.com/paper/2410.23294/) (2024)
- [Enhancing Portfolio Optimization with Deep Learning Insights](https://mlsanity.com/paper/2601.07942/) (2026)

## Abstract (arXiv)

An automatic program that generates constant profit from the financial market is lucrative for every market practitioner. Recent advance in deep reinforcement learning provides a framework toward end-to-end training of such trading agent. In this paper, we propose an Markov Decision Process (MDP) model suitable for the financial trading task and solve it with the state-of-the-art deep recurrent Q-network (DRQN) algorithm. We propose several modifications to the existing learning algorithm to make it more suitable under the financial trading setting, namely 1. We employ a substantially small replay memory (only a few hundreds in size) compared to ones used in modern deep reinforcement learning algorithms (often millions in size.) 2. We develop an action augmentation technique to mitigate the need for random exploration by providing extra feedback signals for all actions to the agent. This enables us to use greedy policy over the course of learning and shows strong empirical performance compared to more commonly used epsilon-greedy exploration. However, this technique is specific to financial trading under a few market assumptions. 3. We sample a longer sequence for recurrent neural network training. A side product of this mechanism is that we can now train the agent for every T steps. This greatly reduces training time since the overall computation is down by a factor of T. We combine all of the above into a complete online learning algorithm and validate our approach on the spot foreign exchange market.