# Explainable Post hoc Portfolio Management Financial Policy of a Deep Reinforcement Learning agent

arXiv: https://arxiv.org/abs/2407.14486 · HTML: https://mlsanity.com/paper/2407.14486/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 25 of its 57 references that are in the map.
- First posted on arXiv 19 Jul 2024.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 25 of 25)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Recent Advances in Reinforcement Learning in Finance](https://mlsanity.com/paper/2112.04553/) (2021)
- [Principles and Practice of Explainable Machine Learning](https://mlsanity.com/paper/2009.11698/) (2020)
- [The Shapley Value in Machine Learning](https://mlsanity.com/paper/2202.05594/) (2022)
- [FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance](https://mlsanity.com/paper/2011.09607/) (2020)
- [Deep Reinforcement Learning for Trading](https://mlsanity.com/paper/1911.10107/) (2019)
- [A Survey of Deep Reinforcement Learning in Video Games](https://mlsanity.com/paper/1912.10944/) (2019)
- [FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance](https://mlsanity.com/paper/2111.09395/) (2021)
- [Practical Deep Reinforcement Learning Approach for Stock Trading](https://mlsanity.com/paper/1811.07522/) (2018)
- [AlphaStock: A Buying-Winners-and-Selling-Losers Investment Strategy using Interpretable Deep Reinforcement Attention Networks](https://mlsanity.com/paper/1908.02646/) (2019)
- [Statistical stability indices for LIME: obtaining reliable explanations for Machine Learning models](https://mlsanity.com/paper/2001.11757/) (2020)
- [Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Review](https://mlsanity.com/paper/2102.04148/) (2021)
- [Explainable Deep Reinforcement Learning: State of the Art and Challenges](https://mlsanity.com/paper/2301.09937/) (2023)
- [Cryptocurrency Portfolio Management with Deep Reinforcement Learning](https://mlsanity.com/paper/1612.01277/) (2016)
- [Deep Hedging of Derivatives Using Reinforcement Learning](https://mlsanity.com/paper/2103.16409/) (2021)
- [Cost-Sensitive Portfolio Selection via Deep Reinforcement Learning](https://mlsanity.com/paper/2003.03051/) (2020)
- [MAPS: Multi-agent Reinforcement Learning-based Portfolio Management System](https://mlsanity.com/paper/2007.05402/) (2020)
- [MSPM: A Modularized and Scalable Multi-Agent Reinforcement Learning-based System for Financial Portfolio Management](https://mlsanity.com/paper/2102.03502/) (2021)
- [Explainable Deep Reinforcement Learning for Portfolio Management: An Empirical Approach](https://mlsanity.com/paper/2111.03995/) (2021)
- [Leveraging Deep Learning and Online Source Sentiment for Financial Portfolio Management](https://mlsanity.com/paper/2309.16679/) (2023)

## Built on by (most cited first, 3 of 3)

- [Deep Reinforcement Learning Agents for Strategic Production Policies in Microeconomic Market Simulations](https://mlsanity.com/paper/2410.20550/) (2024)
- [KASPER: Kolmogorov Arnold Networks for Stock Prediction and Explainable Regimes](https://mlsanity.com/paper/2507.18983/) (2025)
- [Are We Merely Justifying Results ex Post Facto? Quantifying Explanatory Inversion in Post-Hoc Model Explanations](https://mlsanity.com/paper/2504.08919/) (2025)

## Abstract (arXiv)

Financial portfolio management investment policies computed quantitatively by modern portfolio theory techniques like the Markowitz model rely on a set on assumptions that are not supported by data in high volatility markets. Hence, quantitative researchers are looking for alternative models to tackle this problem. Concretely, portfolio management is a problem that has been successfully addressed recently by Deep Reinforcement Learning (DRL) approaches. In particular, DRL algorithms train an agent by estimating the distribution of the expected reward of every action performed by an agent given any financial state in a simulator. However, these methods rely on Deep Neural Networks model to represent such a distribution, that although they are universal approximator models, they cannot explain its behaviour, given by a set of parameters that are not interpretable. Critically, financial investors policies require predictions to be interpretable, so DRL agents are not suited to follow a particular policy or explain their actions. In this work, we developed a novel Explainable Deep Reinforcement Learning (XDRL) approach for portfolio management, integrating the Proximal Policy Optimization (PPO) with the model agnostic explainable techniques of feature importance, SHAP and LIME to enhance transparency in prediction time. By executing our methodology, we can interpret in prediction time the actions of the agent to assess whether they follow the requisites of an investment policy or to assess the risk of following the agent suggestions. To the best of our knowledge, our proposed approach is the first explainable post hoc portfolio management financial policy of a DRL agent. We empirically illustrate our methodology by successfully identifying key features influencing investment decisions, which demonstrate the ability to explain the agent actions in prediction time.