# Model-Free Reinforcement Learning for Asset Allocation

arXiv: https://arxiv.org/abs/2209.10458 · HTML: https://mlsanity.com/paper/2209.10458/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 71 references that are in the map.
- First posted on arXiv 21 Sep 2022.

## Builds on (most cited first, 15 of 15)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Continuous Deep Q-Learning with Model-based Acceleration](https://mlsanity.com/paper/1603.00748/) (2016)
- [A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem](https://mlsanity.com/paper/1706.10059/) (2017)
- [Deep Learning for Financial Applications : A Survey](https://mlsanity.com/paper/2002.05786/) (2020)
- [Deep Learning in Asset Pricing](https://mlsanity.com/paper/1904.00745/) (2019)
- [Cryptocurrency Portfolio Management with Deep Reinforcement Learning](https://mlsanity.com/paper/1612.01277/) (2016)
- [Reinforcement Learning for Portfolio Management](https://mlsanity.com/paper/1909.09571/) (2019)
- [Investor Emotions and Earnings Announcements](https://mlsanity.com/paper/2006.13934/) (2020)
- [Deep Reinforcement Learning for Stock Portfolio Optimization](https://mlsanity.com/paper/2012.06325/) (2020)
- [Using deep Q-learning to understand the tax evasion behavior of risk-averse firms](https://mlsanity.com/paper/1801.09466/) (2018)

## Built on by (most cited first, 1 of 1)

- [Data Cross-Segmentation for Improved Generalization in Reinforcement Learning Based Algorithmic Trading](https://mlsanity.com/paper/2307.09377/) (2023)

## Abstract (arXiv)

Asset allocation (or portfolio management) is the task of determining how to optimally allocate funds of a finite budget into a range of financial instruments/assets such as stocks. This study investigated the performance of reinforcement learning (RL) when applied to portfolio management using model-free deep RL agents. We trained several RL agents on real-world stock prices to learn how to perform asset allocation. We compared the performance of these RL agents against some baseline agents. We also compared the RL agents among themselves to understand which classes of agents performed better. From our analysis, RL agents can perform the task of portfolio management since they significantly outperformed two of the baseline agents (random allocation and uniform allocation). Four RL agents (A2C, SAC, PPO, and TRPO) outperformed the best baseline, MPT, overall. This shows the abilities of RL agents to uncover more profitable trading strategies. Furthermore, there were no significant performance differences between value-based and policy-based RL agents. Actor-critic agents performed better than other types of agents. Also, on-policy agents performed better than off-policy agents because they are better at policy evaluation and sample efficiency is not a significant problem in portfolio management. This study shows that RL agents can substantially improve asset allocation since they outperform strong baselines. On-policy, actor-critic RL agents showed the most promise based on our analysis.