Data to 7 Oct 2026

Graying the black box: Understanding DQNs

arXiv 1602.02658 ↗First posted 8 Feb 2016cs.LG cs.AI cs.NEcited by 68

Not on the map. Why?

Abstract (arXiv)

In recent years there is a growing interest in using deep representations for reinforcement learning. In this paper, we present a methodology and tools to analyze Deep Q-networks (DQNs) in a non-blind matter. Moreover, we propose a new model, the Semi Aggregated Markov Decision Process (SAMDP), and an algorithm that learns it automatically. The SAMDP model allows us to identify spatio-temporal abstractions directly from features and may be used as a sub-goal detector in future work. Using our tools we reveal that the features learned by DQNs aggregate the state space in a hierarchical fashion, explaining its success. Moreover, we are able to understand and describe the policies learned by DQNs for three different Atari2600 games and suggest ways to interpret, debug and optimize deep neural networks in reinforcement learning.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Feb 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 13; 2022: 15; 2023: 13; 2024: 10; 2025: 7; 2026: 9.

68 papers.

Sort:

By area:

2018Scalable agent alignment via reward modeling: a research directioncited by 438
2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
2021Acquisition of Chess Knowledge in AlphaZerocited by 120
2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
2021A Survey on Interpretable Reinforcement Learningcited by 70
2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 41
2021Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learningcited by 38
2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
2022A Survey of Explainable Reinforcement Learningcited by 31
2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
2022Investigating the Properties of Neural Network Representations in Reinforcement Learningcited by 24
2024XAI for All: Can Large Language Models Simplify Explainable AI?cited by 21

38 more

2021Discovering Distinctive "Semantics" in Super-Resolution Networkscited by 20
2023GANterfactual-RL: Understanding Reinforcement Learning Agents' Strategies through Visual Counterfactual Explanationscited by 17
2021Inherently Explainable Reinforcement Learning in Natural Languagecited by 16
2022Explainability in reinforcement learning: perspective and positioncited by 11
2025A Snapshot of Influence: A Local Data Attribution Framework for Online Reinforcement Learningcited by 11
2022MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Treescited by 10
2021Learning Symbolic Rules for Interpretable Deep Reinforcement Learningcited by 9
2022Time-Efficient Reward Learning via Visually Assisted Cluster Rankingcited by 8
2022Lazy-MDPs: Towards Interpretable Reinforcement Learning by Learning When to Actcited by 7
2022Understanding the Evolution of Linear Regions in Deep Reinforcement Learningcited by 7
2021Temporal-Spatial Causal Interpretations for Vision-Based Reinforcement Learningcited by 6
2021Representation Alignment in Neural Networkscited by 6
2025Translate Policy to Language: Flow Matching Generated Rewards for LLM Explanationscited by 6
2021Natural Language Specification of Reinforcement Learning Policies through Differentiable Decision Treescited by 4
2021Benchmarking Perturbation-based Saliency Maps for Explaining Atari Agentscited by 4
2022ProtoX: Explaining a Reinforcement Learning Agent via Prototypingcited by 4
2022Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysiscited by 3
2022A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challengescited by 3
2023AutoDOViz: Human-Centered Automation for Decision Optimizationcited by 3
2023Leveraging Reward Consistency for Interpretable Feature Discovery in Reinforcement Learningcited by 3
2023Advantage Actor-Critic with Reasoner: Explaining the Agent's Behavior from an Exploratory Perspectivecited by 3
2024Benchmarking Deep Reinforcement Learning for Navigation in Denied Sensor Environmentscited by 3
2025Demystifying the Mechanisms Behind Emergent Exploration in Goal-conditioned RLcited by 3
2022Summarising and Comparing Agent Dynamics with Contrastive Spatiotemporal Abstractioncited by 2
2022Reward Learning with Trees: Methods and Evaluationcited by 2
2022Deep Reinforcement Learning with Vector Quantized Encodingcited by 2
2023XAI for time-series classification leveraging image highlight methodscited by 2
2024Abstracted Trajectory Visualization for Explainability in Reinforcement Learningcited by 2
2025Behaviour Discovery and Attribution for Explainable Reinforcement Learningcited by 2
2025TalkToAgent: A Human-centric Explanation of Reinforcement Learning Agents with Large Language Modelscited by 2
2022Neural Distillation as a State Representation Bottleneck in Reinforcement Learningcited by 1
2023FLARE: Fingerprinting Deep Reinforcement Learning Agents using Universal Adversarial Maskscited by 1
2024Which Experiences Are Influential for RL Agents? Efficiently Estimating The Influence of Experiencescited by 1
2024Enhancing Interpretability in Deep Reinforcement Learning through Semantic Clusteringcited by 1
2024Explaining the Unexplained: Revealing Hidden Correlations for Better Interpretabilitycited by 1
2025Learning Nonlinear Causal Reductions to Explain Reinforcement Learning Policiescited by 1
2026Uncovering Latent Phase Structures and Branching Logic in Locomotion Policies: A Case Study on HalfCheetahcited by 1
2026Task-Induced Representational Invariances Depend on Learning Objective in Deep RLcited by 1

50 most cited of 68 All 68 (JSON)