mlsanityData to 14 Oct 2026

Generation of Policy-Level Explanations for Reinforcement Learning

arXiv 1905.12044 ↗First posted 28 May 2019cs.LG cs.AI stat.MLcited by 37

Not on the map. Why?

Abstract (arXiv)

Though reinforcement learning has greatly benefited from the incorporation of neural networks, the inability to verify the correctness of such systems limits their use. Current work in explainable deep learning focuses on explaining only a single decision in terms of input features, making it unsuitable for explaining a sequence of decisions. To address this need, we introduce Abstracted Policy Graphs, which are Markov chains of abstract states. This representation concisely summarizes a policy so that individual decisions can be explained in the context of expected future transitions. Additionally, we propose a method to generate these Abstracted Policy Graphs for deterministic policies given a learned value function and a set of observed transitions, potentially off-policy transitions used during training. Since no restrictions are placed on how the value function is generated, our method is compatible with many existing reinforcement learning methods. We prove that the worst-case time complexity of our method is quadratic in the number of features and linear in the number of provided transitions, O(|F|2 |trsamples|). By applying our method to a family of domains, we show that our method scales well in practice and produces Abstracted Policy Graphs which reliably capture relationships within these domains.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). First posted on arXiv 28 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 11; 2023: 7; 2024: 2; 2025: 8; 2026: 4.

37 papers.

Sort:

By area:

  1. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
  2. 2021A Survey on Interpretable Reinforcement Learningcited by 73
  3. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  4. 2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
  5. 2021Iterative Bounding MDPs: Learning Interpretable Policies via Non-Interpretable Methodscited by 31
  6. 2022A Survey of Explainable Reinforcement Learningcited by 31
  7. 2022Redefining Counterfactual Explanations for Reinforcement Learning: Overview, Challenges and Opportunitiescited by 13
  8. 2022Toward Policy Explanations for Multi-Agent Reinforcement Learningcited by 11
  9. 2021Learning Symbolic Rules for Interpretable Deep Reinforcement Learningcited by 9
  10. 2023Explainable Multi-Agent Reinforcement Learning for Temporal Queriescited by 9
  11. 2023Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examplescited by 9
  12. 2022Lazy-MDPs: Towards Interpretable Reinforcement Learning by Learning When to Actcited by 7
25 more
  1. 2021Feature-Based Interpretable Reinforcement Learning based on State-Transition Modelscited by 6
  2. 2022Explainability Via Causal Self-Talkcited by 6
  3. 2025Translate Policy to Language: Flow Matching Generated Rewards for LLM Explanationscited by 6
  4. 2025A Statistical Case Against Empirical Human-AI Alignmentcited by 6
  5. 2022Local Explanations for Reinforcement Learningcited by 5
  6. 2023Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challengescited by 5
  7. 2025Unveiling the Black Box: A Multi-Layer Framework for Explaining Reinforcement Learning-Based Cyber Agentscited by 4
  8. 2022A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challengescited by 3
  9. 2023Programmatic Imitation Learning from Unlabeled and Noisy Demonstrationscited by 3
  10. 2024ADESSE: Advice Explanations in Complex Repeated Decision-Making Environmentscited by 3
  11. 2022Summarising and Comparing Agent Dynamics with Contrastive Spatiotemporal Abstractioncited by 2
  12. 2022Towards Interpretable Deep Reinforcement Learning Models via Inverse Reinforcement Learningcited by 2
  13. 2024Navigating Trade-offs: Policy Summarization for Multi-Objective Reinforcement Learningcited by 2
  14. 2025Interactive Explanations for Reinforcement-Learning Agentscited by 2
  15. 2025Learning Probabilistic Temporal Logic Specifications for Stochastic Systemscited by 2
  16. 2023XDQN: Inherently Interpretable DQN through Mimickingcited by 1
  17. 2025Learning Nonlinear Causal Reductions to Explain Reinforcement Learning Policiescited by 1
  18. 2026Selecting Decision-Relevant Concepts in Reinforcement Learningcited by 1
  19. 2022Symbolic Distillation for Learned TCP Congestion Controlcited by 0
  20. 2023ASQ-IT: Interactive Explanations for Reinforcement-Learning Agentscited by 0
  21. 2025Strategically Linked Decisions in Long-Term Planning and Reinforcement Learningcited by 0
  22. 2025Explaining Decentralized Multi-Agent Reinforcement Learning Policiescited by 0
  23. 2026TREX: Trajectory Explanations for Multi-Objective Reinforcement Learningcited by 0
  24. 2026BARD-MARL: Byzantine-Agent Detection for Learned Communication in Multi-Agent Reinforcement Learningcited by 0
  25. 2026ORCAID: Oblique Rule-Based Continuous-Action Interpretation for Deep RL Policiescited by 0