# Model-Free Reinforcement Learning for Financial Portfolios: A Brief Survey

arXiv: https://arxiv.org/abs/1904.04973 · HTML: https://mlsanity.com/paper/1904.04973/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2019.

## Built on by (most cited first, 10 of 10)

- [Recent Advances in Reinforcement Learning in Finance](https://mlsanity.com/paper/2112.04553/) (2021)
- [AI in Finance: Challenges, Techniques and Opportunities](https://mlsanity.com/paper/2107.09051/) (2021)
- [Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimization](https://mlsanity.com/paper/2403.07916/) (2024)
- [Systematic Review on Reinforcement Learning in the Field of Fintech](https://mlsanity.com/paper/2305.07466/) (2023)
- [Discrete-Time Mean-Variance Strategy Based on Reinforcement Learning](https://mlsanity.com/paper/2312.15385/) (2023)
- [Profitable Strategy Design by Using Deep Reinforcement Learning for Trades on Cryptocurrency Markets](https://mlsanity.com/paper/2201.05906/) (2022)
- [Racing Towards Reinforcement Learning based control of an Autonomous Formula SAE Car](https://mlsanity.com/paper/2308.13088/) (2023)
- [Pretty darn good control: when are approximate solutions better than approximate models](https://mlsanity.com/paper/2308.13654/) (2023)
- [Introduction to Reinforcement Learning](https://mlsanity.com/paper/2408.07712/) (2024)
- [Onflow: a model free, online portfolio allocation algorithm robust to transaction fees](https://mlsanity.com/paper/2312.05169/) (2023)

## Abstract (arXiv)

Financial portfolio management is one of the problems that are most frequently encountered in the investment industry. Nevertheless, it is not widely recognized that both Kelly Criterion and Risk Parity collapse into Mean Variance under some conditions, which implies that a universal solution to the portfolio optimization problem could potentially exist. In fact, the process of sequential computation of optimal component weights that maximize the portfolio's expected return subject to a certain risk budget can be reformulated as a discrete-time Markov Decision Process (MDP) and hence as a stochastic optimal control, where the system being controlled is a portfolio consisting of multiple investment components, and the control is its component weights. Consequently, the problem could be solved using model-free Reinforcement Learning (RL) without knowing specific component dynamics. By examining existing methods of both value-based and policy-based model-free RL for the portfolio optimization problem, we identify some of the key unresolved questions and difficulties facing today's portfolio managers of applying model-free RL to their investment portfolios.