mlsanityData to 7 Oct 2026

Feature-Based Interpretable Reinforcement Learning based on State-Transition Models

arXiv 2105.07099 ↗First posted 14 May 2021cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Growing concerns regarding the operational usage of AI models in the real-world has caused a surge of interest in explaining AI models' decisions to humans. Reinforcement Learning is not an exception in this regard. In this work, we propose a method for offering local explanations on risk in reinforcement learning. Our method only requires a log of previous interactions between the agent and the environment to create a state-transition model. It is designed to work on RL environments with either continuous or discrete state and action spaces. After creating the model, actions of any agent can be explained in terms of the features most influential in increasing or decreasing risk or any other desirable objective function in the locality of the agent. Through experiments, we demonstrate the effectiveness of the proposed method in providing such explanations.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 19 of its 32 references that are in the map. First posted on arXiv 14 May 2021.

Builds on

19 of 32 references.

References · 19

  1. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  4. 2016OpenAI Gymcited by 1,684
  5. 2018Programmatically Interpretable Reinforcement Learningcited by 192
  6. 2018Verifiable Reinforcement Learning via Policy Extractioncited by 176
  7. 2018SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planningcited by 77
  8. 2017Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approachcited by 68
  9. 2018Toward Interpretable Deep Reinforcement Learning with Linear Model U-Treescited by 65
  10. 2017Interpretable Policies for Reinforcement Learning by Genetic Programmingcited by 47
  11. 2018Contrastive Explanations for Reinforcement Learning in terms of Expected Consequencescited by 44
  12. 2019Generation of Policy-Level Explanations for Reinforcement Learningcited by 37
7 more
  1. 2019Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulationcited by 33
  2. 2019Exploring Computational User Models for Agent Policy Summarizationcited by 29
  3. 2019Conservative Q-Improvement: Reinforcement Learning for an Interpretable Decision-Tree Policycited by 29
  4. 2018Model-Based Reinforcement Learning for Sepsis Treatmentcited by 21
  5. 2016Particle Swarm Optimization for Generating Interpretable Fuzzy Reinforcement Learning Policiescited by 19
  6. 2019Visual Rationalizations in Deep Reinforcement Learning for Atari Gamescited by 8
  7. 2019Optimization Methods for Interpretable Differentiable Decision Trees in Reinforcement Learningcited by 5

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 0; 2024: 0; 2025: 2.

6 papers.

  1. 2022Explainability in reinforcement learning: perspective and positioncited by 11
  2. 2022Targeted Adversarial Attacks on Deep Reinforcement Learning Policies via Model Checkingcited by 5
  3. 2025Turn-based Multi-Agent Reinforcement Learning Model Checkingcited by 3
  4. 2022Summarising and Comparing Agent Dynamics with Contrastive Spatiotemporal Abstractioncited by 2
  5. 2022Disentangled (Un)Controllable Featurescited by 2
  6. 2025Scaling Laws for State Dynamics in Large Language Modelscited by 0