Data to 13 Oct 2026

Visualizing and Understanding Atari Agents

arXiv 1711.00138 ↗First posted 31 Oct 2017cs.AIcited by 141

Not on the map. Why?

Cited by 38 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

While deep reinforcement learning (deep RL) agents are effective at maximizing rewards, it is often unclear what strategies they use to do so. In this paper, we take a step toward explaining deep RL agents through a case study using Atari 2600 environments. In particular, we focus on using saliency maps to understand how an agent learns and executes a policy. We introduce a method for generating useful saliency maps and use it to show 1) what strong agents attend to, 2) whether agents are making decisions for the right or wrong reasons, and 3) how agents evolve during learning. We also test our method on non-expert human subjects and find that it improves their ability to reason about these agents. Overall, our results show that saliency information can provide significant insight into an RL agent's decisions and learning behavior.

Key facts. Cited by 141 papers in the map (data through 13 Oct 2026). First posted on arXiv 31 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2018: 1; 2019: 3; 2020: 0; 2021: 29; 2022: 24; 2023: 23; 2024: 17; 2025: 26; 2026: 18.

141 papers.

Sort:

By area:

2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,368
2018Scalable agent alignment via reward modeling: a research directioncited by 443
2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
2021Acquisition of Chess Knowledge in AlphaZerocited by 122
2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 118
2021A Survey on Interpretable Reinforcement Learningcited by 70
2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 42
2021Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learningcited by 38
2021Counterfactual Explanations in Sequential Decision Making Under Uncertaintycited by 36
2021Symbols as a Lingua Franca for Bridging Human-AI Chasm for Explainable and Advisable AI Systemscited by 36
2021Domain Adaptation In Reinforcement Learning Via Latent Unified State Representationcited by 33
2022Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learningcited by 33

38 more

2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
2021Iterative Bounding MDPs: Learning Interpretable Policies via Non-Interpretable Methodscited by 31
2022A Survey of Explainable Reinforcement Learningcited by 31
2024Quantization-Aware Imitation-Learning for Resource-Efficient Robotic Controlcited by 25
2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
2021Collective eXplainable AI: Explaining Cooperative Strategies and Agent Contribution in Multiagent Reinforcement Learning with Shapley Valuescited by 24
2022Investigating the Properties of Neural Network Representations in Reinforcement Learningcited by 24
2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 24
2023Causal Reinforcement Learning: A Surveycited by 22
2021ExoMiner: A Highly Accurate and Explainable Deep Learning Classifier that Validates 301 New Exoplanetscited by 17
2024End-to-End Neuro-Symbolic Reinforcement Learning with Textual Explanationscited by 17
2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
2021A Game-Theoretic Taxonomy of Visual Concepts in DNNscited by 16
2023Explaining Reinforcement Learning Agents Through Counterfactual Action Outcomescited by 13
2021Critic Guided Segmentation of Rewarding Objects in First-Person Viewscited by 12
2022Redefining Counterfactual Explanations for Reinforcement Learning: Overview, Challenges and Opportunitiescited by 12
2024Hindsight PRIORs for Reward Learning from Human Preferencescited by 12
2022Explainability in reinforcement learning: perspective and positioncited by 11
2022Explainable Reinforcement Learning via Model Transformscited by 11
2023Aligning Robot and Human Representationscited by 11
2025Saliency-Aware Quantized Imitation Learning for Efficient Robotic Controlcited by 11
2025A Snapshot of Influence: A Local Data Attribution Framework for Online Reinforcement Learningcited by 11
2021Discovering User-Interpretable Capabilities of Black-Box Planning Agentscited by 10
2024Understanding Individual Agent Importance in Multi-Agent System via Counterfactual Reasoningcited by 10
2026R2-Dreamer: Redundancy-Reduced World Models without Decoders or Augmentationcited by 10
2021Training a Resilient Q-Network against Observational Interferencecited by 9
2021Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learningcited by 9
2021Learning Symbolic Rules for Interpretable Deep Reinforcement Learningcited by 9
2021Explaining Deep Reinforcement Learning Agents In The Atari Domain through a Surrogate Modelcited by 9
2021Goal-Aware Cross-Entropy for Multi-Target Reinforcement Learningcited by 7
2021Towards Safe, Explainable, and Regulated Autonomous Drivingcited by 7
2022Explaining Reinforcement Learning Policies through Counterfactual Trajectoriescited by 7
2022Lazy-MDPs: Towards Interpretable Reinforcement Learning by Learning When to Actcited by 7
2023A Closer Look at Reward Decomposition for High-Level Robotic Explanationscited by 7
2023Explaining RL Decisions with Trajectoriescited by 7
2024Safety Implications of Explainable Artificial Intelligence in End-to-End Autonomous Drivingcited by 7
2021"I Don't Think So": Summarizing Policy Disagreements for Agent Comparisoncited by 6
2021Temporal-Spatial Causal Interpretations for Vision-Based Reinforcement Learningcited by 6

50 most cited of 141 All 141 (JSON)