mlsanityData to 14 Oct 2026

Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learning

arXiv 2103.04067 ↗First posted 6 Mar 2021cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (DRL) has great potential for acquiring the optimal action in complex environments such as games and robot control. However, it is difficult to analyze the decision-making of the agent, i.e., the reasons it selects the action acquired by learning. In this work, we propose Mask-Attention A3C (Mask A3C), which introduces an attention mechanism into Asynchronous Advantage Actor-Critic (A3C), which is an actor-critic-based DRL method, and can analyze the decision-making of an agent in DRL. A3C consists of a feature extractor that extracts features from an image, a policy branch that outputs the policy, and a value branch that outputs the state value. In this method, we focus on the policy and value branches and introduce an attention mechanism into them. The attention mechanism applies a mask processing to the feature maps of each branch using mask-attention that expresses the judgment reason for the policy and state value with a heat map. We visualized mask-attention maps for games on the Atari 2600 and found we could easily analyze the reasons behind an agent's decision-making in various game tasks. Furthermore, experimental results showed that the agent could achieve a higher performance by introducing the attention mechanism.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 26 of its 32 references that are in the map. First posted on arXiv 6 Mar 2021.

Builds on

26 of 32 references.

References · 26

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2015Trust Region Policy Optimizationcited by 2,580
  6. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  8. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  9. 2016OpenAI Gymcited by 1,736
  10. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  11. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  12. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
14 more
  1. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  2. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  3. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  4. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
  5. 2017Visualizing and Understanding Atari Agentscited by 141
  6. 2018Attention Branch Network: Learning of Attention Mechanism for Visual Explanationcited by 109
  7. 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
  8. 2015Massively Parallel Methods for Deep Reinforcement Learningcited by 82
  9. 2019Towards Interpretable Reinforcement Learning Using Attention Augmented Agentscited by 82
  10. 2019A Survey of Deep Reinforcement Learning in Video Gamescited by 73
  11. 2018AGIL: Learning Attention from Human for Visuomotor Taskscited by 32
  12. 2017Deep Learning for Video Game Playingcited by 30
  13. 2015Deep Attention Recurrent Q-Networkcited by 27
  14. 2019Reinforcement Learning with Attention that Works: A Self-Supervised Approachcited by 19

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 0; 2024: 3; 2025: 4.

9 papers.

  1. 2022A Survey of Explainable Reinforcement Learningcited by 31
  2. 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agentscited by 23
  3. 2024Layer-Wise Relevance Propagation with Conservation Property for ResNetcited by 7
  4. 2024ACTER: Diverse and Actionable Counterfactual Sequences for Explaining and Diagnosing RL Policiescited by 4
  5. 2025AEGIS: Human Attention-based Explainable Guidance for Intelligent Vehicle Systemscited by 2
  6. 2025Revealing Human Attention Patterns from Gameplay Analysis for Reinforcement Learningcited by 1
  7. 2022Interpretable Deep Reinforcement Learning for Green Security Games with Real-Time Informationcited by 0
  8. 2025OPTIC-ER: A Reinforcement Learning Framework for Real-Time Emergency Response and Equitable Resource Allocation in Underserved African Communitiescited by 0
  9. 2025Attention Lattice Adapter: Visual Explanation Generation for Visual Foundation Modelcited by 0