# Explainable Deep Reinforcement Learning for Portfolio Management: An Empirical Approach

arXiv: https://arxiv.org/abs/2111.03995 · HTML: https://mlsanity.com/paper/2111.03995/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 7 Nov 2021.

## Builds on (most cited first, 15 of 15)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI](https://mlsanity.com/paper/1907.07374/) (2019)
- [Explainable Reinforcement Learning Through a Causal Lens](https://mlsanity.com/paper/1905.10958/) (2019)
- [Grad-CAM: Why did you say that?](https://mlsanity.com/paper/1611.07450/) (2016)
- [Sanity Checks for Saliency Metrics](https://mlsanity.com/paper/1912.01451/) (2019)
- [Explainability in Deep Reinforcement Learning](https://mlsanity.com/paper/2008.06693/) (2020)
- [FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance](https://mlsanity.com/paper/2011.09607/) (2020)
- [Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learning](https://mlsanity.com/paper/1912.05743/) (2019)
- [Multi-Period Trading via Convex Optimization](https://mlsanity.com/paper/1705.00109/) (2017)

## Built on by (most cited first, 5 of 5)

- [Dynamic Datasets and Market Environments for Financial Reinforcement Learning](https://mlsanity.com/paper/2304.13174/) (2023)
- [Explainable Post hoc Portfolio Management Financial Policy of a Deep Reinforcement Learning agent](https://mlsanity.com/paper/2407.14486/) (2024)
- [Leveraging Deep Learning and Online Source Sentiment for Financial Portfolio Management](https://mlsanity.com/paper/2309.16679/) (2023)
- [An Ensemble Method of Deep Reinforcement Learning for Automated Cryptocurrency Trading](https://mlsanity.com/paper/2309.00626/) (2023)
- [MIGT: Memory Instance Gated Transformer Framework for Financial Portfolio Management](https://mlsanity.com/paper/2502.07280/) (2025)

## Abstract (arXiv)

Deep reinforcement learning (DRL) has been widely studied in the portfolio management task. However, it is challenging to understand a DRL-based trading strategy because of the black-box nature of deep neural networks. In this paper, we propose an empirical approach to explain the strategies of DRL agents for the portfolio management task. First, we use a linear model in hindsight as the reference model, which finds the best portfolio weights by assuming knowing actual stock returns in foresight. In particular, we use the coefficients of a linear model in hindsight as the reference feature weights. Secondly, for DRL agents, we use integrated gradients to define the feature weights, which are the coefficients between reward and features under a linear regression model. Thirdly, we study the prediction power in two cases, single-step prediction and multi-step prediction. In particular, we quantify the prediction power by calculating the linear correlations between the feature weights of a DRL agent and the reference feature weights, and similarly for machine learning methods. Finally, we evaluate a portfolio management task on Dow Jones 30 constituent stocks during 01/01/2009 to 09/01/2021. Our approach empirically reveals that a DRL agent exhibits a stronger multi-step prediction power than machine learning methods.