Real Eyes Realize Faster: Gaze Stability and Pupil Novelty for Efficient Egocentric Learning
Not on the map. Why?
Abstract (arXiv)
Always-on egocentric cameras are increasingly used as demonstrations for embodied robotics, imitation learning, and assistive AR, but the resulting video streams are dominated by redundant and low-quality frames. Under the storage and battery constraints of wearable devices, choosing which frames to keep is as important as how to learn from them. We observe that modern eye-tracking headsets provide a continuous, training-free side channel that decomposes into two complementary axes: gaze fixation captures visual stability (quality), while pupil response captures arousal-linked moments (novelty). We operationalize this insight as a Dual-Criterion Frame Curator that first gates frames by gaze quality and then ranks the survivors by pupil-derived novelty. On the Visual Experience Dataset (VEDB), curated frames at 10% budget match the classification performance of the full stream, and naive signal fusion consistently destroys both contributions. The benefit is task-dependent: pupil ranking improves activity recognition, while gaze-only selection already dominates for scene recognition, confirming that the two signals serve genuinely different roles. Our method requires no model inference and operates at capture time, offering a path toward efficient, always-on egocentric data curation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 48 references that are in the map. First posted on arXiv 4 Mar 2026.
Builds on
18 of 48 references.
References · 18
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2016Understanding intermediate layers using linear classifier probescited by 1,279
- 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
- 2023DataComp: In search of the next generation of multimodal datasetscited by 635
- 2020Rescaling Egocentric Visioncited by 529
- 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 337
- 2023Data-centric Artificial Intelligence: A Surveycited by 202
- 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 201
- 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
- 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 160
6 more
- 2016Video Summarization with Long Short-term Memorycited by 142
- 2018Predicting Gaze in Egocentric Video by Learning Task-dependent Attention Transitioncited by 78
- 2022In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimationcited by 51
- 2020Integrating Human Gaze into Attention for Egocentric Activity Recognitioncited by 42
- 2025FOCUS: Efficient Keyframe Selection for Long Video Understandingcited by 30
- 2024The Visual Experience Dataset: Over 200 Recorded Hours of Integrated Eye Movement, Odometry, and Egocentric Videocited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ARGaze: Autoregressive Transformers for Online Egocentric Gaze Estimation
- 2026EgoGazeLite: On-Device Egocentric Gaze Prediction for Token-Efficient Multimodal LLM Video Input
- 2026GazeFlow: From Human Gaze Behavior to Generative Egocentric Gaze Prediction
- 2025Gaze-VLM:Bridging Gaze and VLMs through Attention Regularization for Egocentric Understanding
- 2025From My View to Yours: Learning Egocentric Cues from Exocentric Video using Privileged Egocentric Supervision
- 2026TAVIS: A Benchmark for Egocentric Active Vision and Anticipatory Gaze in Imitation Learning
- 2026Gaze-Regularized VLMs for Ego-Centric Behavior Understanding
- 2025HAGI++: Head-Assisted Gaze Imputation and Generation