mlsanityData to 7 Oct 2026

Attention Trajectories as a Diagnostic Axis for Deep Reinforcement Learning

arXiv 2511.20591 ↗First posted 25 Nov 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The emergence and evolution of feature reliance in deep reinforcement learning agents remain poorly understood. Here, we introduce a methodological framework for analyzing the learning process through quantitative analysis of saliency maps. This approach aggregates saliency information at the object and modality level into hierarchical attention profiles, quantifying how agents allocate attention over time, thereby forming attention trajectories throughout training. These profiles are then compared across controlled conditions, connected to behavioral measurements and reproduced with different saliency methods to assess the robustness of the findings. Applied to Atari 2600 benchmarks, custom Pong environments, and biomechanical user simulations in visuomotor tasks, this framework uncovers algorithm-specific attention biases, diagnosed unintended reward-driven strategies, and overfitting to redundant sensory channels. These patterns correspond to measurable behavioral differences, demonstrating empirical links between attention profiles, learning dynamics, and agent behavior. The results establish attention trajectories as a promising diagnostic axis for tracing how feature reliance develops during training and for identifying biases and vulnerabilities invisible to performance metrics alone.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 63 references that are in the map. First posted on arXiv 25 Nov 2025.

Builds on

38 of 63 references.

References · 38

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2023Segment Anythingcited by 8,158
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  6. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  7. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  8. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  9. 2018Sanity Checks for Saliency Mapscited by 1,149
  10. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  11. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  12. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
26 more
  1. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  2. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 359
  3. 2015Explaining NonLinear Classification Decisions with Deep Taylor Decompositioncited by 248
  4. 2019Unsupervised State Representation Learning in Ataricited by 144
  5. 2017Visualizing and Understanding Atari Agentscited by 141
  6. 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
  7. 2019Observational Overfitting in Reinforcement Learningcited by 90
  8. 2019Towards Interpretable Reinforcement Learning Using Attention Augmented Agentscited by 82
  9. 2019Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learningcited by 60
  10. 2021Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAycited by 53
  11. 2019Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attributioncited by 50
  12. 2018Transparency and Explanation in Deep Reinforcement Learning Neural Networkscited by 40
  13. 2020Local and Global Explanations of Agent Behavior: Integrating Strategy Summaries with Saliency Mapscited by 30
  14. 2021Toward Explainable AI for Regression Modelscited by 30
  15. 2025Mechanistic understanding and validation of large AI models with SemanticLenscited by 29
  16. 2018Towards Better Interpretability in Deep Q-Networkscited by 22
  17. 2021Revisiting Sanity Checks for Saliency Mapscited by 20
  18. 2020Machine versus Human Attention in Deep Reinforcement Learning Taskscited by 16
  19. 2019Free-Lunch Saliency via Attention in Atari Agentscited by 12
  20. 2019Visual Rationalizations in Deep Reinforcement Learning for Atari Gamescited by 8
  21. 2024A Fresh Look at Sanity Checks for Saliency Mapscited by 8
  22. 2018Learn to Interpret Atari Agentscited by 6
  23. 2024SIM2VR: Towards Automated Biomechanical Testing in VRcited by 6
  24. 2021Benchmarking Perturbation-based Saliency Maps for Explaining Atari Agentscited by 4
  25. 2025Demystifying Reward Design in Reinforcement Learning for Upper Extremity Interaction: Practical Guidelines for Biomechanical Simulations in HCIcited by 3
  26. 2022Deep Apprenticeship Learning for Playing Gamescited by 1

Built on by

Not cited yet.