mlsanityData to 7 Oct 2026

Mutual Context Network for Jointly Estimating Egocentric Gaze and Actions

arXiv 1901.01874 ↗First posted 7 Jan 2019cs.CVcited by 54

Not on the map. Why?

Abstract (arXiv)

In this work, we address two coupled tasks of gaze prediction and action recognition in egocentric videos by exploring their mutual context. Our assumption is that in the procedure of performing a manipulation task, what a person is doing determines where the person is looking at, and the gaze point reveals gaze and non-gaze regions which contain important and complementary information about the undergoing action. We propose a novel mutual context network (MCN) that jointly learns action-dependent gaze prediction and gaze-guided action recognition in an end-to-end manner. Experiments on public egocentric video datasets demonstrate that our MCN achieves state-of-the-art performance of both gaze prediction and action recognition.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 8; 2023: 6; 2024: 7; 2025: 19; 2026: 7.

54 papers.

Sort:

By area:

  1. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understandingcited by 106
  2. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 93
  3. 2021Goal-Oriented Gaze Estimation for Zero-Shot Learningcited by 68
  4. 2022In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimationcited by 51
  5. 2021With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognitioncited by 41
  6. 2023Memory-and-Anticipation Transformer for Online Action Understandingcited by 39
  7. 2021Predicting the Future from First Person (Egocentric) Vision: A Surveycited by 36
  8. 2023LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuningcited by 28
  9. 2025EgoExoBench: A Benchmark for First- and Third-person View Video Understanding in MLLMscited by 22
  10. 2025In the Eye of MLLM: Benchmarking Egocentric Video Intent Understanding with Gaze-Guided Promptingcited by 21
  11. 2023Listen to Look into the Future: Audio-Visual Egocentric Gaze Anticipationcited by 20
  12. 2025Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learningcited by 20
38 more
  1. 2025Challenges and Trends in Egocentric Vision: A Surveycited by 20
  2. 2022RadioTransformer: A Cascaded Global-Focal Transformer for Visual Attention-guided Disease Classificationcited by 19
  3. 2022Egocentric Video Task Translationcited by 19
  4. 2024Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Modelcited by 19
  5. 2023Egocentric Auditory Attention Localization in Conversationscited by 18
  6. 2024Masked Video and Body-worn IMU Autoencoder for Egocentric Action Recognitioncited by 18
  7. 2024EAGLE: Egocentric AGgregated Language-video Enginecited by 16
  8. 2023Fine-grained Affordance Annotation for Egocentric Hand-Object Interaction Videoscited by 15
  9. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
  10. 2025Gaze-VLM:Bridging Gaze and VLMs through Attention Regularization for Egocentric Understandingcited by 10
  11. 2025UniLS: End-to-End Audio-Driven Avatars for Unified Listening and Speakingcited by 10
  12. 2024A Backpack Full of Skills: Egocentric Video Understanding with Diverse Task Perspectivescited by 9
  13. 2022SOS! Self-supervised Learning Over Sets Of Handled Objects In Egocentric Action Recognitioncited by 8
  14. 2024GPT4Ego: Unleashing the Potential of Pre-trained Models for Zero-Shot Egocentric Action Recognitioncited by 8
  15. 2025Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Visioncited by 8
  16. 2025Towards Interactive Intelligence for Digital Humanscited by 8
  17. 2025An Egocentric Vision-Language Model based Portable Real-time Smart Assistantcited by 7
  18. 2025VideoTG-R1: Boosting Video Temporal Grounding via Curriculum Reinforcement Learning on Reflected Boundary Annotationscited by 7
  19. 2026Gaze-Regularized Vision-Language-Action Models for Robotic Manipulationcited by 7
  20. 2021Learning to Recognize Actions on Objects in Egocentric Video with Attention Dictionariescited by 6
  21. 2021Leveraging Human Selective Attention for Medical Image Analysis with Limited Training Datacited by 5
  22. 2025Hier-EgoPack: Hierarchical Egocentric Video Understanding with Diverse Task Perspectivescited by 5
  23. 2025ANNEXE: Unified Analyzing, Answering, and Pixel Grounding for Egocentric Interactioncited by 5
  24. 2021Stacked Temporal Attention: Improving First-person Action Recognition by Emphasizing Discriminative Clipscited by 4
  25. 2025EgoAdapt: Adaptive Multisensory Distillation and Policy Learning for Efficient Egocentric Perceptioncited by 4
  26. 2025Ego-centric Predictive Model Conditioned on Hand Trajectoriescited by 4
  27. 2025SkillSight: Efficient First-Person Skill Assessment with Gazecited by 4
  28. 2022Context-empowered Visual Attention Prediction in Pedestrian Scenarioscited by 2
  29. 2023Proposal-based Temporal Action Localization with Point-level Supervisioncited by 2
  30. 2025Weakly Supervised Temporal Sentence Grounding via Positive Sample Miningcited by 2
  31. 2026Vinci2: Providing Proactive Assistance in Continuous Egocentric Videoscited by 2
  32. 2022Human Gaze Guided Attention for Surgical Activity Recognitioncited by 1
  33. 2025SFHand: Learning Embodied Manipulation by Streaming Egocentric 3D Hand Forecastingcited by 1
  34. 2022Precise Affordance Annotation for Egocentric Action Video Datasetscited by 0
  35. 2022A Survey on Human Action Recognitioncited by 0
  36. 2025Living the Novel: A System for Generating Self-Training Timeline-Aware Conversational Agents from Novelscited by 0
  37. 2025DETACH : Decomposed Spatio-Temporal Alignment for Exocentric Video and Ambient Sensors with Staged Learningcited by 0
  38. 2026Robust Egocentric Visual Attention Prediction Through Language-guided Scene Context-aware Learningcited by 0

50 most cited of 54 All 54 (JSON)