mlsanityData to 7 Oct 2026

Predicting Gaze in Egocentric Video by Learning Task-dependent Attention Transition

arXiv 1803.09125 ↗First posted 24 Mar 2018cs.CVcited by 78

Not on the map. Why?

Abstract (arXiv)

We present a new computational model for gaze prediction in egocentric videos by exploring patterns in temporal shift of gaze fixations (attention transition) that are dependent on egocentric manipulation tasks. Our assumption is that the high-level context of how a task is completed in a certain way has a strong influence on attention transition and should be modeled for gaze prediction in natural dynamic scenes. Specifically, we propose a hybrid model based on deep neural networks which integrates task-dependent attention transition with bottom-up saliency prediction. In particular, the task-dependent attention transition is learned with a recurrent neural network to exploit the temporal context of gaze fixations, e.g. looking at a cup after moving gaze away from a grasped bottle. Experiments on public egocentric activity datasets show that our model significantly outperforms state-of-the-art gaze prediction methods and is able to learn meaningful transition of human attention.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 8; 2023: 13; 2024: 12; 2025: 24; 2026: 12.

78 papers.

Sort:

By area:

  1. 2023Affordance Diffusion: Synthesizing Hand-Object Interactionscited by 110
  2. 2021Ego-Exo: Transferring Visual Representations from Third-person to First-person Videoscited by 98
  3. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 93
  4. 2023An Outlook into the Future of Egocentric Visioncited by 84
  5. 2022Learning Affordance Grounding from Exocentric Imagescited by 82
  6. 2022My View is the Best View: Procedure Learning from Egocentric Videoscited by 81
  7. 2021Goal-Oriented Gaze Estimation for Zero-Shot Learningcited by 68
  8. 2024RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesiscited by 55
  9. 2022In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimationcited by 51
  10. 2024EgoVideo: Exploring Egocentric Foundation Model and Downstream Adaptationcited by 44
  11. 2024Retrieval-Augmented Egocentric Video Captioningcited by 42
  12. 2023Memory-and-Anticipation Transformer for Online Action Understandingcited by 39
38 more
  1. 2021Predicting the Future from First Person (Egocentric) Vision: A Surveycited by 36
  2. 2022Compound Prototype Matching for Few-shot Action Recognitioncited by 35
  3. 2023Affordance Grounding from Demonstration Video to Target Imagecited by 34
  4. 2023LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuningcited by 28
  5. 2021Learning Visual Affordance Grounding from Demonstration Videoscited by 25
  6. 2023LvBench: A Benchmark for Long-form Video Understanding with Versatile Multi-modal Question Answeringcited by 25
  7. 2025EgoExoBench: A Benchmark for First- and Third-person View Video Understanding in MLLMscited by 22
  8. 2022Grounded Affordance from Exocentric Viewcited by 21
  9. 2025In the Eye of MLLM: Benchmarking Egocentric Video Intent Understanding with Gaze-Guided Promptingcited by 21
  10. 2021ST-MTL: Spatio-Temporal Multitask Learning Model to Predict Scanpath While Tracking Instruments in Robotic Surgerycited by 20
  11. 2023Listen to Look into the Future: Audio-Visual Egocentric Gaze Anticipationcited by 20
  12. 2023Learning from Semantic Alignment between Unpaired Multiviews for Egocentric Video Recognitioncited by 20
  13. 2025Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learningcited by 20
  14. 2024Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Modelcited by 19
  15. 2025EgoExo-Gen: Ego-centric Video Prediction by Watching Exo-centric Videoscited by 19
  16. 2023Egocentric Auditory Attention Localization in Conversationscited by 18
  17. 2024Masked Video and Body-worn IMU Autoencoder for Egocentric Action Recognitioncited by 18
  18. 2023XAIR: A Framework of Explainable AI in Augmented Realitycited by 17
  19. 2023Fine-grained Affordance Annotation for Egocentric Hand-Object Interaction Videoscited by 15
  20. 2024Gazing Into Missteps: Leveraging Eye-Gaze for Unsupervised Mistake Detection in Egocentric Videos of Skilled Human Activitiescited by 15
  21. 2024INTRA: Interaction Relationship-aware Weakly Supervised Affordance Groundingcited by 12
  22. 2024Do Egocentric Video-Language Models Truly Understand Hand-Object Interactions?cited by 11
  23. 2025egoPPG: Heart Rate Estimation from Eye-Tracking Cameras in Egocentric Systems to Benefit Downstream Vision Taskscited by 11
  24. 2025Where, What, Why: Towards Explainable Driver Attention Predictioncited by 10
  25. 2025UniLS: End-to-End Audio-Driven Avatars for Unified Listening and Speakingcited by 10
  26. 2025EgoM2P: Egocentric Multimodal Multitask Pretrainingcited by 9
  27. 2026EgoForge: Goal-Directed Egocentric World Simulatorcited by 9
  28. 2025Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Visioncited by 8
  29. 2025Towards Interactive Intelligence for Digital Humanscited by 8
  30. 2026Towards Multimodal Lifelong Understanding: A Dataset and Agentic Baselinecited by 8
  31. 2024CaRe-Ego: Contact-aware Relationship Modeling for Egocentric Interactive Hand-object Segmentationcited by 7
  32. 2025An Egocentric Vision-Language Model based Portable Real-time Smart Assistantcited by 7
  33. 2021Leveraging Human Selective Attention for Medical Image Analysis with Limited Training Datacited by 5
  34. 2021Indoor Future Person Localization from an Egocentric Wearable Cameracited by 4
  35. 2021Stacked Temporal Attention: Improving First-person Action Recognition by Emphasizing Discriminative Clipscited by 4
  36. 2025Closed-Loop Transfer for Weakly-supervised Affordance Groundingcited by 4
  37. 2025SkillSight: Efficient First-Person Skill Assessment with Gazecited by 4
  38. 2022Self-supervised Video-centralised Transformer for Video Face Clusteringcited by 3

50 most cited of 78 All 78 (JSON)