mlsanityData to 14 Oct 2026

Revealing Human Attention Patterns from Gameplay Analysis for Reinforcement Learning

arXiv 2504.11118 ↗First posted 15 Apr 2025cs.LG cs.CVcited by 1

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

This study introduces a novel method for revealing human internal attention patterns (decision-relevant attention) from gameplay data alone, leveraging offline attention techniques from reinforcement learning (RL). We propose contextualized, task-relevant (CTR) attention networks, which generate attention maps from both human and RL agent gameplay in Atari environments. To evaluate whether the human CTR maps reveal internal attention patterns, we validate our model by quantitative and qualitative comparison to the agent maps as well as to a temporally integrated overt attention (TIOA) model based on human eye-tracking data. Our results show that human CTR maps are more sparse than the agent ones and align better with the TIOA maps. Following a qualitative visual comparison we conclude that they likely capture patterns of internal attention. As a further application, we use these maps to guide RL agents, finding that human attention-guided agents achieve slightly improved and more stable learning compared to baselines, and significantly outperform TIOA-based agents. This work advances the understanding of human-agent attention differences and provides a new approach for extracting and validating internal attention patterns from behavioral data.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 24 of its 39 references that are in the map. First posted on arXiv 15 Apr 2025.

Builds on

24 of 39 references.

References · 24

  1. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  2. 2015Prioritized Experience Replaycited by 1,099
  3. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  4. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  5. 2019Making Convolutional Networks Shift-Invariant Againcited by 391
  6. 2014Recurrent Models of Visual Attentioncited by 349
  7. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 307
  8. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 303
  9. 2017Visualizing and Understanding Atari Agentscited by 141
  10. 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 122
  11. 2019Unsupervised Learning of Object Keypoints for Perception and Controlcited by 109
  12. 2019Towards Interpretable Reinforcement Learning Using Attention Augmented Agentscited by 82
12 more
  1. 2020Munchausen Reinforcement Learningcited by 69
  2. 2020Predicting Goal-directed Human Attention Using Inverse Reinforcement Learningcited by 53
  3. 2019Leveraging Human Guidance for Deep Reinforcement Learning Taskscited by 42
  4. 2022Visual Attention Methods in Deep Learning: An In-Depth Surveycited by 37
  5. 2018AGIL: Learning Attention from Human for Visuomotor Taskscited by 32
  6. 2019Reinforcement Learning with Attention that Works: A Self-Supervised Approachcited by 19
  7. 2020Machine versus Human Attention in Deep Reinforcement Learning Taskscited by 16
  8. 2020Unsupervised Object Keypoint Learning using Local Spatial Predictabilitycited by 13
  9. 2019Free-Lunch Saliency via Attention in Atari Agentscited by 12
  10. 2021Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learningcited by 9
  11. 2021Temporal-Spatial Causal Interpretations for Vision-Based Reinforcement Learningcited by 6
  12. 2018Visual Attention for Behavioral Cloning in Autonomous Drivingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Estimating Central, Peripheral, and Temporal Visual Contributions to Human Decision Making in Atari Gamescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Attention Trajectories as a Diagnostic Axis for Deep Reinforcement Learning
  2. 2024Revealing the Learning Process in Reinforcement Learning Agents Through Attention-Oriented Metrics
  3. 2019Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset ↗
  4. 2020Selective Eye-gaze Augmentation To Enhance Imitation Learning In Atari Games
  5. 2024From Transformer to Biology: A Hierarchical Model for Attention in Complex Problem-Solving ↗
  6. 2025Learning to Focus: Prioritizing Informative Histories with Structured Attention Mechanisms in Partially Observable Reinforcement Learning
  7. 2025Interpretable Learning Dynamics in Unsupervised Reinforcement Learning
  8. 2020Efficiently Guiding Imitation Learning Agents with Human Gaze