# Model-based Deep Reinforcement Learning for Dynamic Portfolio Optimization

arXiv: https://arxiv.org/abs/1901.08740 · HTML: https://mlsanity.com/paper/1901.08740/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jan 2019.

## Built on by (most cited first, 22 of 22)

- [Recent Advances in Reinforcement Learning in Finance](https://mlsanity.com/paper/2112.04553/) (2021)
- [Reinforcement Learning for Quantitative Trading](https://mlsanity.com/paper/2109.13851/) (2021)
- [DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities](https://mlsanity.com/paper/2201.09058/) (2021)
- [A Review of Reinforcement Learning in Financial Applications](https://mlsanity.com/paper/2411.12746/) (2024)
- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning](https://mlsanity.com/paper/2407.07631/) (2024)
- [Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimization](https://mlsanity.com/paper/2403.07916/) (2024)
- [Deep Reinforcement Learning and Mean-Variance Strategies for Responsible Portfolio Optimization](https://mlsanity.com/paper/2403.16667/) (2024)
- [FinWorld: An All-in-One Open-Source Platform for End-to-End Financial AI Research and Deployment](https://mlsanity.com/paper/2508.02292/) (2025)
- [Markov Decision Processes under Model Uncertainty](https://mlsanity.com/paper/2206.06109/) (2022)
- [Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Markets](https://mlsanity.com/paper/2301.08688/) (2023)
- [Reinforcement Learning Applied to Trading Systems: A Survey](https://mlsanity.com/paper/2212.06064/) (2022)
- [A novel multi-agent dynamic portfolio optimization learning system based on hierarchical deep reinforcement learning](https://mlsanity.com/paper/2501.06832/) (2025)
- [Predicting Liquidity-Aware Bond Yields using Causal GANs and Deep Reinforcement Learning with LLM Evaluation](https://mlsanity.com/paper/2502.17011/) (2025)
- [Decision by Supervised Learning with Deep Ensembles: A Practical Framework for Robust Portfolio Optimization](https://mlsanity.com/paper/2503.13544/) (2025)
- [Deep Reinforcement Learning for Optimal Asset Allocation Using DDPG with TiDE](https://mlsanity.com/paper/2508.20103/) (2025)
- [Stock Trading Optimization through Model-based Reinforcement Learning with Resistance Support Relative Strength](https://mlsanity.com/paper/2205.15056/) (2022)
- [Reinforcement Learning for Financial Index Tracking](https://mlsanity.com/paper/2308.02820/) (2023)
- [Deep Reinforcement Learning in Factor Investment](https://mlsanity.com/paper/2509.16206/) (2025)
- [Robust Reinforcement Learning in Finance: Modeling Market Impact with Elliptic Uncertainty Sets](https://mlsanity.com/paper/2510.19950/) (2025)
- [The Enhanced Physics-Informed Kolmogorov-Arnold Networks: Applications of Newton's Laws in Financial Deep Reinforcement Learning (RL) Algorithms](https://mlsanity.com/paper/2602.01388/) (2026)
- [Deep Reinforcement Learning for Reliability Based Bi-Objective Portfolio Optimization](https://mlsanity.com/paper/2607.06610/) (2026)

## Abstract (arXiv)

Dynamic portfolio optimization is the process of sequentially allocating wealth to a collection of assets in some consecutive trading periods, based on investors' return-risk profile. Automating this process with machine learning remains a challenging problem. Here, we design a deep reinforcement learning (RL) architecture with an autonomous trading agent such that, investment decisions and actions are made periodically, based on a global objective, with autonomy. In particular, without relying on a purely model-free RL agent, we train our trading agent using a novel RL architecture consisting of an infused prediction module (IPM), a generative adversarial data augmentation module (DAM) and a behavior cloning module (BCM). Our model-based approach works with both on-policy or off-policy RL algorithms. We further design the back-testing and execution engine which interact with the RL agent in real time. Using historical em real financial market data, we simulate trading with practical constraints, and demonstrate that our proposed model is robust, profitable and risk-sensitive, as compared to baseline trading strategies and model-free RL agents from prior work.