mlsanityData to 14 Oct 2026

REVEAL-IT: REinforcement learning with Visibility of Evolving Agent poLicy for InTerpretability

arXiv 2406.14214 ↗First posted 20 Jun 2024cs.AIcited by 0

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

Understanding the agent's learning process, particularly the factors that contribute to its success or failure post-training, is crucial for comprehending the rationale behind the agent's decision-making process. Prior methods clarify the learning process by creating a structural causal model (SCM) or visually representing the distribution of value functions. Nevertheless, these approaches have constraints as they exclusively function in 2D-environments or with uncomplicated transition dynamics. Understanding the agent's learning process in complicated environments or tasks is more challenging. In this paper, we propose REVEAL-IT, a novel framework for explaining the learning process of an agent in complex environments. Initially, we visualize the policy structure and the agent's learning process for various training tasks. By visualizing these findings, we can understand how much a particular training task or stage affects the agent's performance in test. Then, a GNN-based explainer learns to highlight the most important section of the policy, providing a more clear and robust explanation of the agent's learning process. The experiments demonstrate that explanations derived from this framework can effectively help in the optimization of the training tasks, resulting in improved learning efficiency and final performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 32 of its 49 references that are in the map. First posted on arXiv 20 Jun 2024.

Builds on

32 of 49 references.

References · 32

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Graph Attention Networkscited by 8,923
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2015Continuous control with deep reinforcement learningcited by 3,407
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  7. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  8. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  9. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  10. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  11. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  12. 2017A Distributional Perspective on Reinforcement Learningcited by 765
20 more
  1. 2019GNNExplainer: Generating Explanations for Graph Neural Networkscited by 737
  2. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  3. 2020Parameterized Explainer for Graph Neural Networkcited by 426
  4. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  5. 2018TextWorld: A Learning Environment for Text-based Gamescited by 310
  6. 2021On Explainability of Graph Neural Networks via Subgraph Explorationscited by 281
  7. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  8. 2017Visualizing and Understanding Atari Agentscited by 141
  9. 2020Explainable Reinforcement Learning: A Surveycited by 141
  10. 2020Explainability in Deep Reinforcement Learningcited by 97
  11. 2019Towards Interpretable Reinforcement Learning Using Attention Augmented Agentscited by 82
  12. 2020An empirical investigation of the challenges of real-world reinforcement learningcited by 77
  13. 2020Semiparametric proximal causal inferencecited by 59
  14. 2020Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learningcited by 53
  15. 2020Causally Correct Partial Models for Reinforcement Learningcited by 24
  16. 2023MixupExplainer: Generalizing Explanations for Graph Neural Networks with Data Augmentationcited by 18
  17. 2021A Meta-Learning Approach for Training Explainable Graph Neural Networkscited by 15
  18. 2020Interactive Visualization for Debugging RLcited by 5
  19. 2023Curriculum Reinforcement Learning via Morphology-Environment Co-Evolutioncited by 5
  20. 2023Safe and Sample-efficient Reinforcement Learning for Clustered Dynamic Environmentscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Explainable Reinforcement Learning via a Causal World Model
  2. 2025Interpretable Learning Dynamics in Unsupervised Reinforcement Learning
  3. 2025Explainable Reinforcement Learning via Temporal Policy Decomposition
  4. 2025A Survey of Explainable Reinforcement Learning: Targets, Methods and Needs
  5. 2024Towards a Research Community in Interpretable Reinforcement Learning: the InterpPol Workshop
  6. 2024Abstracted Trajectory Visualization for Explainability in Reinforcement Learning
  7. 2025Explainable Reinforcement Learning Agents Using World Models
  8. 2025Behaviour Discovery and Attribution for Explainable Reinforcement Learning