# Fidelity-Induced Interpretable Policy Extraction for Reinforcement Learning

arXiv: https://arxiv.org/abs/2309.06097 · HTML: https://mlsanity.com/paper/2309.06097/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 38 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 21 of 21)

- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI](https://mlsanity.com/paper/1907.07374/) (2019)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning](https://mlsanity.com/paper/1711.00832/) (2017)
- [Google Research Football: A Novel Reinforcement Learning Environment](https://mlsanity.com/paper/1907.11180/) (2019)
- [Distilling a Neural Network Into a Soft Decision Tree](https://mlsanity.com/paper/1711.09784/) (2017)
- [A Divergence Minimization Perspective on Imitation Learning Methods](https://mlsanity.com/paper/1911.02256/) (2019)
- [Verifiable Reinforcement Learning via Policy Extraction](https://mlsanity.com/paper/1805.08328/) (2018)
- [Explainable Reinforcement Learning: A Survey](https://mlsanity.com/paper/2005.06247/) (2020)
- [Measuring Neural Net Robustness with Constraints](https://mlsanity.com/paper/1605.07262/) (2016)
- [Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees](https://mlsanity.com/paper/1807.05887/) (2018)
- [Explainability in Human-Agent Systems](https://mlsanity.com/paper/1904.08123/) (2019)
- [Mo"ET: Mixture of Expert Trees and its Application to Verifiable Reinforcement Learning](https://mlsanity.com/paper/1906.06717/) (2019)
- [MAVIPER: Learning Decision Tree Policies for Interpretable Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2205.12449/) (2022)
- [Proxy Experience Replay: Federated Distillation for Distributed Reinforcement Learning](https://mlsanity.com/paper/2005.06105/) (2020)
- [Deep Reinforcement Learning for Black-Box Testing of Android Apps](https://mlsanity.com/paper/2101.02636/) (2021)
- [Explainable Action Advising for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2211.07882/) (2022)
- [Designing Interpretable Approximations to Deep Reinforcement Learning](https://mlsanity.com/paper/2010.14785/) (2020)
- [Complementary reinforcement learning towards explainable agents](https://mlsanity.com/paper/1901.00188/) (2019)
- [Neural-to-Tree Policy Distillation with Policy Improvement Criterion](https://mlsanity.com/paper/2108.06898/) (2021)

## Abstract (arXiv)

Deep Reinforcement Learning (DRL) has achieved remarkable success in sequential decision-making problems. However, existing DRL agents make decisions in an opaque fashion, hindering the user from establishing trust and scrutinizing weaknesses of the agents. While recent research has developed Interpretable Policy Extraction (IPE) methods for explaining how an agent takes actions, their explanations are often inconsistent with the agent's behavior and thus, frequently fail to explain. To tackle this issue, we propose a novel method, Fidelity-Induced Policy Extraction (FIPE). Specifically, we start by analyzing the optimization mechanism of existing IPE methods, elaborating on the issue of ignoring consistency while increasing cumulative rewards. We then design a fidelity-induced mechanism by integrate a fidelity measurement into the reinforcement learning feedback. We conduct experiments in the complex control environment of StarCraft II, an arena typically avoided by current IPE methods. The experiment results demonstrate that FIPE outperforms the baselines in terms of interaction performance and consistency, meanwhile easy to understand.