Interpretable Learning Dynamics in Unsupervised Reinforcement Learning
On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL
Abstract (arXiv)
We present an interpretability framework for unsupervised reinforcement learning (URL) agents, aimed at understanding how intrinsic motivation shapes attention, behavior, and representation learning. We analyze five agents DQN, RND, ICM, PPO, and a Transformer-RND variant trained on procedurally generated environments, using Grad-CAM, Layer-wise Relevance Propagation (LRP), exploration metrics, and latent space clustering. To capture how agents perceive and adapt over time, we introduce two metrics: attention diversity, which measures the spatial breadth of focus, and attention change rate, which quantifies temporal shifts in attention. Our findings show that curiosity-driven agents display broader, more dynamic attention and exploratory behavior than their extrinsically motivated counterparts. Among them, TransformerRND combines wide attention, high exploration coverage, and compact, structured latent representations. Our results highlight the influence of architectural inductive biases and training signals on internal agent dynamics. Beyond reward-centric evaluation, the proposed framework offers diagnostic tools to probe perception and abstraction in RL agents, enabling more interpretable and generalizable behavior.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 2 of its 2 references that are in the map. First posted on arXiv 6 May 2025.
Builds on
2 of 2 references.
References · 2
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
- 2018Exploration by Random Network Distillationcited by 993
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Mechanistic Interpretability of Reinforcement Learning Agents
- 2024Towards a Research Community in Interpretable Reinforcement Learning: the InterpPol Workshop
- 2024Revealing the Learning Process in Reinforcement Learning Agents Through Attention-Oriented Metrics
- 2026Escaping Local Views: Discovering Latent Concepts for Interpretable Multi-Agent Reinforcement Learning
- 2024REVEAL-IT: REinforcement learning with Visibility of Evolving Agent poLicy for InTerpretability
- 2026What Do World Models Learn in RL? Probing Latent Representations in Learned Environment Simulators
- 2025Perspectives for Direct Interpretability in Multi-Agent Deep Reinforcement Learning
- 2025Attention Trajectories as a Diagnostic Axis for Deep Reinforcement Learning