# A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem

arXiv: https://arxiv.org/abs/1706.10059 · HTML: https://mlsanity.com/paper/1706.10059/

## Key facts

- Cited by 128 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2017.

## Built on by (most cited first, 30 of 128)

- [Recent Advances in Reinforcement Learning in Finance](https://mlsanity.com/paper/2112.04553/) (2021)
- [Large Language Model Agent in Financial Trading: A Survey](https://mlsanity.com/paper/2408.06361/) (2024)
- [FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance](https://mlsanity.com/paper/2111.09395/) (2021)
- [Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey](https://mlsanity.com/paper/2110.01411/) (2021)
- [Reinforcement Learning for Quantitative Trading](https://mlsanity.com/paper/2109.13851/) (2021)
- [Bayesian Bellman Operators](https://mlsanity.com/paper/2106.05012/) (2021)
- [DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities](https://mlsanity.com/paper/2201.09058/) (2021)
- [Deep Graph Convolutional Reinforcement Learning for Financial Portfolio Management -- DeepPocket](https://mlsanity.com/paper/2105.08664/) (2021)
- [A Review of Reinforcement Learning in Financial Applications](https://mlsanity.com/paper/2411.12746/) (2024)
- [TradExpert: Revolutionizing Trading with Mixture of Expert LLMs](https://mlsanity.com/paper/2411.00782/) (2024)
- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [MSPM: A Modularized and Scalable Multi-Agent Reinforcement Learning-based System for Financial Portfolio Management](https://mlsanity.com/paper/2102.03502/) (2021)
- [MetaTrader: An Reinforcement Learning Approach Integrating Diverse Policies for Portfolio Optimization](https://mlsanity.com/paper/2210.01774/) (2022)
- [Prediction Arena: Benchmarking AI Models on Real-World Prediction Markets](https://mlsanity.com/paper/2604.07355/) (2026)
- [Deep Reinforcement Learning Approach for Trading Automation in The Stock Market](https://mlsanity.com/paper/2208.07165/) (2022)
- [From Deep Learning to LLMs: A survey of AI in Quantitative Investment](https://mlsanity.com/paper/2503.21422/) (2025)
- [A policy gradient approach for Finite Horizon Constrained Markov Decision Processes](https://mlsanity.com/paper/2210.04527/) (2022)
- [Reinforcement Learning with Maskable Stock Representation for Portfolio Management in Customizable Stock Pools](https://mlsanity.com/paper/2311.10801/) (2023)
- [LLM-Enhanced Black-Litterman Portfolio Optimization](https://mlsanity.com/paper/2504.14345/) (2025)
- [No Prior Mask: Eliminate Redundant Action for Deep Reinforcement Learning](https://mlsanity.com/paper/2312.06258/) (2023)
- [Combining Transformer based Deep Reinforcement Learning with Black-Litterman Model for Portfolio Optimization](https://mlsanity.com/paper/2402.16609/) (2024)
- [PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets](https://mlsanity.com/paper/2302.00586/) (2023)
- [Developing A Multi-Agent and Self-Adaptive Framework with Deep Reinforcement Learning for Dynamic Portfolio Risk Management](https://mlsanity.com/paper/2402.00515/) (2024)
- [Dynamic data generation and dynamic portfolio selection: an application of a score-based diffusion model](https://mlsanity.com/paper/2507.09916/) (2025)
- [A Prescriptive Dirichlet Power Allocation Policy with Deep Reinforcement Learning](https://mlsanity.com/paper/2201.08445/) (2022)
- [Auto.gov: Learning-based Governance for Decentralized Finance (DeFi)](https://mlsanity.com/paper/2302.09551/) (2023)
- [Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimization](https://mlsanity.com/paper/2403.07916/) (2024)
- [Deep Reinforcement Learning and Mean-Variance Strategies for Responsible Portfolio Optimization](https://mlsanity.com/paper/2403.16667/) (2024)
- [A Single-Loop Robust Policy Gradient Method for Robust Markov Decision Processes](https://mlsanity.com/paper/2406.00274/) (2024)
- [FinWorld: An All-in-One Open-Source Platform for End-to-End Financial AI Research and Deployment](https://mlsanity.com/paper/2508.02292/) (2025)

## Abstract (arXiv)

Financial portfolio management is the process of constant redistribution of a fund into different financial products. This paper presents a financial-model-free Reinforcement Learning framework to provide a deep machine learning solution to the portfolio management problem. The framework consists of the Ensemble of Identical Independent Evaluators (EIIE) topology, a Portfolio-Vector Memory (PVM), an Online Stochastic Batch Learning (OSBL) scheme, and a fully exploiting and explicit reward function. This framework is realized in three instants in this work with a Convolutional Neural Network (CNN), a basic Recurrent Neural Network (RNN), and a Long Short-Term Memory (LSTM). They are, along with a number of recently reviewed or published portfolio-selection strategies, examined in three back-test experiments with a trading period of 30 minutes in a cryptocurrency market. Cryptocurrencies are electronic and decentralized alternatives to government-issued money, with Bitcoin as the best-known example of a cryptocurrency. All three instances of the framework monopolize the top three positions in all experiments, outdistancing other compared trading algorithms. Although with a high commission rate of 0.25% in the backtests, the framework is able to achieve at least 4-fold returns in 50 days.