mlsanityData to 7 Oct 2026

Towards Interpretable Reinforcement Learning Using Attention Augmented Agents

arXiv 1906.02500 ↗First posted 6 Jun 2019cs.LG stat.MLcited by 82

Not on the map. Why?

Abstract (arXiv)

Inspired by recent work in attention models for image captioning and question answering, we present a soft attention model for the reinforcement learning domain. This model uses a soft, top-down attention mechanism to create a bottleneck in the agent, forcing it to focus on task-relevant information by sequentially querying its view of the environment. The output of the attention mechanism allows direct observation of the information used by the agent to select its actions, enabling easier interpretation of this model than of traditional models. We analyze different strategies that the agents learn and show that a handful of strategies arise repeatedly across different games. We also show that the model learns to query separately about space and content (`where' vs. `what'). We demonstrate that an agent using this mechanism can achieve performance competitive with state-of-the-art models on ATARI tasks while still being interpretable.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 20; 2022: 22; 2023: 14; 2024: 6; 2025: 15; 2026: 5.

82 papers.

Sort:

By area:

  1. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
  2. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  3. 2021Going Beyond Linear Transformers with Recurrent Fast Weight Programmerscited by 84
  4. 2021A Survey on Interpretable Reinforcement Learningcited by 70
  5. 2021Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learningcited by 38
  6. 2022Visual Attention Methods in Deep Learning: An In-Depth Surveycited by 37
  7. 2022The Neural Race Reduction: Dynamics of Abstraction in Gated Networkscited by 37
  8. 2022Deep Transformer Q-Networks for Partially Observable Reinforcement Learningcited by 35
  9. 2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
  10. 2022A Survey of Explainable Reinforcement Learningcited by 31
  11. 2021Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Controlcited by 24
  12. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 23
38 more
  1. 2023Causal Reinforcement Learning: A Surveycited by 22
  2. 2021Self-Interpretable Model with TransformationEquivariant Interpretationcited by 21
  3. 2023Exploring the Promise and Limits of Real-Time Recurrent Learningcited by 19
  4. 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
  5. 2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
  6. 2024Graph Reinforcement Learning for Combinatorial Optimization: A Survey and Unifying Perspectivecited by 16
  7. 2022Learning to Generalize with Object-centric Agents in the Open World Survival Game Craftercited by 13
  8. 2025A Snapshot of Influence: A Local Data Attribution Framework for Online Reinforcement Learningcited by 11
  9. 2021Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learningcited by 9
  10. 2021Active Sensing for Communications by Learningcited by 9
  11. 2023Learning to Identify Critical States for Reinforcement Learning from Videoscited by 9
  12. 2023Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examplescited by 9
  13. 2021Joint Attention for Multi-Agent Coordination and Social Learningcited by 8
  14. 2021Goal-Aware Cross-Entropy for Multi-Target Reinforcement Learningcited by 7
  15. 2021Agent-Centric Representations for Multi-Agent Reinforcement Learningcited by 6
  16. 2021Temporal-Spatial Causal Interpretations for Vision-Based Reinforcement Learningcited by 6
  17. 2022Toward Causal-Aware RL: State-Wise Action-Refined Temporal Differencecited by 6
  18. 2022Explainability Via Causal Self-Talkcited by 6
  19. 2024Policy-shaped prediction: avoiding distractions in model-based reinforcement learningcited by 6
  20. 2022Video Violence Recognition and Localization Using a Semi-Supervised Hard Attention Modelcited by 5
  21. 2022Local Explanations for Reinforcement Learningcited by 5
  22. 2022Reinforcement Learning in Practice: Opportunities and Challengescited by 5
  23. 2021Causal Analysis of Agent Behavior for AI Safetycited by 4
  24. 2021Design Strategy Network: A deep hierarchical framework to represent generative design strategies in complex action spacescited by 4
  25. 2022On learning history based policies for controlling Markov decision processescited by 4
  26. 2022ProtoX: Explaining a Reinforcement Learning Agent via Prototypingcited by 4
  27. 2023Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learningcited by 4
  28. 2022Priors, Hierarchy, and Information Asymmetry for Skill Transfer in Reinforcement Learningcited by 3
  29. 2022Policy Regularization for Legible Behaviorcited by 3
  30. 2023Leveraging Reward Consistency for Interpretable Feature Discovery in Reinforcement Learningcited by 3
  31. 2023Advantage Actor-Critic with Reasoner: Explaining the Agent's Behavior from an Exploratory Perspectivecited by 3
  32. 2025Embodied Navigation with Auxiliary Task of Action Description Predictioncited by 3
  33. 2021Pretrained Encoders are All You Needcited by 2
  34. 2021Identifying Reasoning Flaws in Planning-Based RL Using Tree Explanationscited by 2
  35. 2022Deep Reinforcement Learning with Vector Quantized Encodingcited by 2
  36. 2024Model-based Policy Optimization using Symbolic World Modelcited by 2
  37. 2025Deep Reinforcement Learning via Object-Centric Attentioncited by 2
  38. 2025AEGIS: Human Attention-based Explainable Guidance for Intelligent Vehicle Systemscited by 2

50 most cited of 82 All 82 (JSON)