# Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Prediction States

arXiv: https://arxiv.org/abs/2002.05780 · HTML: https://mlsanity.com/paper/2002.05780/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2020.

## Built on by (most cited first, 30 of 55)

- [A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist](https://mlsanity.com/paper/2402.18485/) (2024)
- [CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing](https://mlsanity.com/paper/2106.09292/) (2021)
- [DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregation](https://mlsanity.com/paper/2103.11109/) (2021)
- [Universal Trading for Order Execution with Oracle Policy Distillation](https://mlsanity.com/paper/2103.10860/) (2021)
- [Reinforcement Learning for Quantitative Trading](https://mlsanity.com/paper/2109.13851/) (2021)
- [FLAG-Trader: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Trading](https://mlsanity.com/paper/2502.11433/) (2025)
- [Solving General Natural-Language-Description Optimization Problems with Large Language Models](https://mlsanity.com/paper/2407.07924/) (2024)
- [DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities](https://mlsanity.com/paper/2201.09058/) (2021)
- [LiveTradeBench: Seeking Real-World Alpha with Large Language Models](https://mlsanity.com/paper/2511.03628/) (2025)
- [TradExpert: Revolutionizing Trading with Mixture of Expert LLMs](https://mlsanity.com/paper/2411.00782/) (2024)
- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [MSPM: A Modularized and Scalable Multi-Agent Reinforcement Learning-based System for Financial Portfolio Management](https://mlsanity.com/paper/2102.03502/) (2021)
- [MetaTrader: An Reinforcement Learning Approach Integrating Diverse Policies for Portfolio Optimization](https://mlsanity.com/paper/2210.01774/) (2022)
- [Learning Multi-Agent Intention-Aware Communication for Optimal Multi-Order Execution in Finance](https://mlsanity.com/paper/2307.03119/) (2023)
- [Reinforcement Learning with Maskable Stock Representation for Portfolio Management in Customizable Stock Pools](https://mlsanity.com/paper/2311.10801/) (2023)
- [Alpha-R1: Alpha Screening with LLM Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2512.23515/) (2025)
- [Combining Transformer based Deep Reinforcement Learning with Black-Litterman Model for Portfolio Optimization](https://mlsanity.com/paper/2402.16609/) (2024)
- [MANA-Net: Mitigating Aggregated Sentiment Homogenization with News Weighting for Enhanced Market Prediction](https://mlsanity.com/paper/2409.05698/) (2024)
- [Modeling News Interactions and Influence for Financial Market Prediction](https://mlsanity.com/paper/2410.10614/) (2024)
- [MILLION: A General Multi-Objective Framework with Controllable Risk for Portfolio Management](https://mlsanity.com/paper/2412.03038/) (2024)
- [Multi-Agent Reinforcement Learning with Shared Resources for Inventory Management](https://mlsanity.com/paper/2212.07684/) (2022)
- [PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets](https://mlsanity.com/paper/2302.00586/) (2023)
- [Developing A Multi-Agent and Self-Adaptive Framework with Deep Reinforcement Learning for Dynamic Portfolio Risk Management](https://mlsanity.com/paper/2402.00515/) (2024)
- [Deep Reinforcement Learning and Mean-Variance Strategies for Responsible Portfolio Optimization](https://mlsanity.com/paper/2403.16667/) (2024)
- [Learning Multiple Initial Solutions to Optimization Problems](https://mlsanity.com/paper/2411.02158/) (2024)
- [Asset Allocation: From Markowitz to Deep Reinforcement Learning](https://mlsanity.com/paper/2208.07158/) (2022)
- [Intelligent Trading Systems: A Sentiment-Aware Reinforcement Learning Approach](https://mlsanity.com/paper/2112.02095/) (2021)
- [Applications of Reinforcement Learning in Finance -- Trading with a Double Deep Q-Network](https://mlsanity.com/paper/2206.14267/) (2022)
- [Reinforcement Learning Applied to Trading Systems: A Survey](https://mlsanity.com/paper/2212.06064/) (2022)
- [STORM: A Spatio-Temporal Factor Model Based on Dual Vector Quantized Variational Autoencoders for Financial Trading](https://mlsanity.com/paper/2412.09468/) (2024)

## Abstract (arXiv)

Portfolio management (PM) is a fundamental financial planning task that aims to achieve investment goals such as maximal profits or minimal risks. Its decision process involves continuous derivation of valuable information from various data sources and sequential decision optimization, which is a prospective research direction for reinforcement learning (RL). In this paper, we propose SARL, a novel State-Augmented RL framework for PM. Our framework aims to address two unique challenges in financial PM: (1) data heterogeneity -- the collected information for each asset is usually diverse, noisy and imbalanced (e.g., news articles); and (2) environment uncertainty -- the financial market is versatile and non-stationary. To incorporate heterogeneous data and enhance robustness against environment uncertainty, our SARL augments the asset information with their price movement prediction as additional states, where the prediction can be solely based on financial data (e.g., asset prices) or derived from alternative sources such as news. Experiments on two real-world datasets, (i) Bitcoin market and (ii) HighTech stock market with 7-year Reuters news articles, validate the effectiveness of SARL over existing PM approaches, both in terms of accumulated profits and risk-adjusted profits. Moreover, extensive simulations are conducted to demonstrate the importance of our proposed state augmentation, providing new insights and boosting performance significantly over standard RL-based PM method and other baselines.