# Reinforcement Learning for Portfolio Management

arXiv: https://arxiv.org/abs/1909.09571 · HTML: https://mlsanity.com/paper/1909.09571/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2019.

## Built on by (most cited first, 12 of 12)

- [AI in Finance: Challenges, Techniques and Opportunities](https://mlsanity.com/paper/2107.09051/) (2021)
- [Risk-Sensitive RL for Alleviating Exploration Dilemmas in Large Language Models](https://mlsanity.com/paper/2509.24261/) (2025)
- [Towards Efficient Risk-Sensitive Policy Gradient: An Iteration Complexity Analysis](https://mlsanity.com/paper/2403.08955/) (2024)
- [Optimal Market Making by Reinforcement Learning](https://mlsanity.com/paper/2104.04036/) (2021)
- [Markov Decision Processes under Model Uncertainty](https://mlsanity.com/paper/2206.06109/) (2022)
- [Model-Free Reinforcement Learning for Asset Allocation](https://mlsanity.com/paper/2209.10458/) (2022)
- [Adaptive Road Configurations for Improved Autonomous Vehicle-Pedestrian Interactions using Reinforcement Learning](https://mlsanity.com/paper/2303.12289/) (2023)
- [Deep Reinforcement Learning and Convex Mean-Variance Optimisation for Portfolio Management](https://mlsanity.com/paper/2203.11318/) (2022)
- [A Novel Experts Advice Aggregation Framework Using Deep Reinforcement Learning for Portfolio Management](https://mlsanity.com/paper/2212.14477/) (2022)
- [Cryptocurrency Portfolio Optimization by Neural Networks](https://mlsanity.com/paper/2310.01148/) (2023)
- [Dynamic Portfolio Optimization via Augmented DDPG with Quantum Price Levels-Based Trading Strategy](https://mlsanity.com/paper/2501.08528/) (2025)
- [Robust Reinforcement Learning in Finance: Modeling Market Impact with Elliptic Uncertainty Sets](https://mlsanity.com/paper/2510.19950/) (2025)

## Abstract (arXiv)

In this thesis, we develop a comprehensive account of the expressive power, modelling efficiency, and performance advantages of so-called trading agents (i.e., Deep Soft Recurrent Q-Network (DSRQN) and Mixture of Score Machines (MSM)), based on both traditional system identification (model-based approach) as well as on context-independent agents (model-free approach). The analysis provides conclusive support for the ability of model-free reinforcement learning methods to act as universal trading agents, which are not only capable of reducing the computational and memory complexity (owing to their linear scaling with the size of the universe), but also serve as generalizing strategies across assets and markets, regardless of the trading universe on which they have been trained. The relatively low volume of daily returns in financial market data is addressed via data augmentation (a generative approach) and a choice of pre-training strategies, both of which are validated against current state-of-the-art models. For rigour, a risk-sensitive framework which includes transaction costs is considered, and its performance advantages are demonstrated in a variety of scenarios, from synthetic time-series (sinusoidal, sawtooth and chirp waves), simulated market series (surrogate data based), through to real market data (S&P 500 and EURO STOXX 50). The analysis and simulations confirm the superiority of universal model-free reinforcement learning agents over current portfolio management model in asset allocation strategies, with the achieved performance advantage of as much as 9.2% in annualized cumulative returns and 13.4% in annualized Sharpe Ratio.