# Eyes on Target: Gaze-Aware Object Detection in Egocentric Video

arXiv: https://arxiv.org/abs/2511.01237 · HTML: https://mlsanity.com/paper/2511.01237/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 3 Nov 2025.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Integrating Human Gaze into Attention for Egocentric Activity Recognition](https://mlsanity.com/paper/2011.03920/) (2020)
- [How Does Attention Work in Vision Transformers? A Visual Analytics Attempt](https://mlsanity.com/paper/2303.13731/) (2023)
- [Gaze Embeddings for Zero-Shot Image Classification](https://mlsanity.com/paper/1611.09309/) (2016)
- [GazeGNN: A Gaze-Guided Graph Neural Network for Chest X-ray Classification](https://mlsanity.com/paper/2305.18221/) (2023)
- [Human Attention in Fine-grained Classification](https://mlsanity.com/paper/2111.01628/) (2021)
- [Gaze-directed Vision GNN for Mitigating Shortcut Learning in Medical Image](https://mlsanity.com/paper/2406.14050/) (2024)
- [Weakly Supervised Attended Object Detection Using Gaze Data as Annotations](https://mlsanity.com/paper/2204.07090/) (2022)

## Built on by (most cited first, 2 of 2)

- [VEGAS: Human-Aligned Video Caption Evaluation via Gaze](https://mlsanity.com/paper/2607.08489/) (2026)
- [G3Ego: Gaze-Guided Graphs for Egocentric Action Understanding](https://mlsanity.com/paper/2608.20157/) (2026)

## Abstract (arXiv)

Human gaze offers rich supervisory signals for understanding visual attention in complex visual environments. In this paper, we propose Eyes on Target, a novel depth-aware and gaze-guided object detection framework designed for egocentric videos. Our approach injects gaze-derived features into the attention mechanism of a Vision Transformer (ViT), effectively biasing spatial feature selection toward human-attended regions. Unlike traditional object detectors that treat all regions equally, our method emphasises viewer-prioritised areas to enhance object detection. We validate our method on an egocentric simulator dataset where human visual attention is critical for task assessment, illustrating its potential in evaluating human performance in simulation scenarios. We evaluate the effectiveness of our gaze-integrated model through extensive experiments and ablation studies, demonstrating consistent gains in detection accuracy over gaze-agnostic baselines on both the custom simulator dataset and public benchmarks, including Ego4D Ego-Motion and Ego-CH-Gaze datasets. To interpret model behaviour, we also introduce a gaze-aware attention head importance metric, revealing how gaze cues modulate transformer attention dynamics.