mlsanityData to 14 Oct 2026

AGIL: Learning Attention from Human for Visuomotor Tasks

arXiv 1806.03960 ↗First posted 1 Jun 2018cs.CV cs.AI cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

When intelligent agents learn visuomotor behaviors from human demonstrations, they may benefit from knowing where the human is allocating visual attention, which can be inferred from their gaze. A wealth of information regarding intelligent decision making is conveyed by human gaze allocation; hence, exploiting such information has the potential to improve the agents' performance. With this motivation, we propose the AGIL (Attention Guided Imitation Learning) framework. We collect high-quality human action and gaze data while playing Atari games in a carefully controlled experimental setting. Using these data, we first train a deep neural network that can predict human gaze positions and visual attention with high accuracy (the gaze network) and then train another network to predict human actions (the policy network). Incorporating the learned attention model from the gaze network into the policy network significantly improves the action prediction accuracy and task performance.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). First posted on arXiv 1 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 2; 2020: 2; 2021: 9; 2022: 5; 2023: 4; 2024: 2; 2025: 4; 2026: 4.

32 papers.

Sort:

By area:

  1. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  2. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  3. 2021DRIVE: Deep Reinforced Accident Anticipation with Visual Explanationcited by 46
  4. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 36
  5. 2019MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learningcited by 29
  6. 2023Retina : Low-Power Eye Tracking with Event Camera and Spiking Hardwarecited by 27
  7. 2021Gaze-based dual resolution deep imitation learning for high-precision dexterous robot manipulationcited by 17
  8. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
  9. 2019Gaze Training by Modulated Dropout Improves Imitation Learningcited by 10
  10. 2021Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learningcited by 9
  11. 2022Memory-based gaze prediction in deep imitation learning for robot manipulationcited by 9
  12. 2026Gaze-Regularized Vision-Language-Action Models for Robotic Manipulationcited by 9
20 more
  1. 2023NOIR: Neural Signal Operated Intelligent Robots for Everyday Activitiescited by 8
  2. 2022Measuring Human Perception to Improve Open Set Recognitioncited by 7
  3. 2020Efficiently Guiding Imitation Learning Agents with Human Gazecited by 6
  4. 2020Selective Eye-gaze Augmentation To Enhance Imitation Learning In Atari Gamescited by 6
  5. 2022Using Human Perception to Regularize Transfer Learningcited by 6
  6. 2021Imitation Learning with Human Eye Gaze via Multi-Objective Predictioncited by 4
  7. 2024Diverse Policies Recovering via Pointwise Mutual Information Weighted Imitation Learningcited by 4
  8. 2025AutoFocus-IL: VLM-based Saliency Maps for Data-Efficient Visual Imitation Learning without Extra Human Annotationscited by 4
  9. 2022Guiding Machine Perception with Psychophysicscited by 3
  10. 2025GABRIL: Gaze-Based Regularization for Mitigating Causal Confusion in Imitation Learningcited by 3
  11. 2021Pretrained Encoders are All You Needcited by 2
  12. 2021Towards Interactive Reinforcement Learning with Intrinsic Feedbackcited by 2
  13. 2025AEGIS: Human Attention-based Explainable Guidance for Intelligent Vehicle Systemscited by 2
  14. 2021Trajectory-Constrained Deep Latent Visual Attention for Improved Local Planning in Presence of Heterogeneous Terraincited by 1
  15. 2025Revealing Human Attention Patterns from Gameplay Analysis for Reinforcement Learningcited by 1
  16. 2023Int-HRL: Towards Intention-based Hierarchical Reinforcement Learningcited by 0
  17. 2023Action Q-Transformer: Visual Explanation in Deep Reinforcement Learning with Encoder-Decoder Model using Action Querycited by 0
  18. 2026Estimating Central, Peripheral, and Temporal Visual Contributions to Human Decision Making in Atari Gamescited by 0
  19. 2026Neurosymbolic Imitation Learning with Human Guidance: A Privileged Information Approachcited by 0
  20. 2026Gaze Prompts: Temporally Dense Human Attention for Vision-Language-Action Fine-Tuningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Visual Attention Prediction Improves Performance of Autonomous Drone Racing Agents
  2. 2021Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control
  3. 2024IDIL: Imitation Learning of Intent-Driven Expert Behavior
  4. 2020Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning
  5. 2019Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset ↗
  6. 2026Fast Human Attention Prediction for Fixation-guided Active Perception in Autonomous Navigation
  7. 2024A Transformer-Based Model for the Prediction of Human Gaze Behavior on Videos
  8. 2019Utilizing Eye Gaze to Enhance the Generalization of Imitation Networks to Unseen Environments