Motion-Based Tokenization for Cross-Dataset Egocentric Gaze Modeling
Not on the map. Why?
Abstract (arXiv)
Gaze is increasingly used as an input signal for vision and multimodal models, yet no consensus exists on how to represent it across datasets. Raw traces preserve detail but are noisy and device-dependent, while coarse event labels are easy to model but can discard local motion structure. We formulate event-aligned, fixed-horizon angular displacement as an interpretable, event-conditioned motion vocabulary and compare it with event-only, spatial, absolute-angle, learned vector-quantized, and continuous representations. To assess transfer alongside target predictability and token collapse, our evaluation combines next-token prediction with target-domain regret, low-order target references, paired bootstrap, order sensitivity, motif overlap, and frozen structural probes. In an event-aligned headset benchmark, angular-motion tokens have lower target-domain regret than frozen-codebook VQ tokens in one transfer direction, while the reverse direction is inconclusive. The probes reveal complementary representation properties, and event-only tokens show that low perplexity can retain little motion information. On a third egocentric dataset, a matched comparison of I-VT, native, and frame-span interfaces shows that event construction materially changes transfer: native events have the lowest regret into EGTEA, while frame-span events have zero motif overlap and fail severely as a source. Motion-based tokenization therefore provides a compact representation for event-aligned egocentric gaze streams, while the evaluation identifies how target predictability and event construction shape cross-dataset conclusions.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 12 of its 29 references that are in the map. First posted on arXiv 24 Aug 2026.
Builds on
12 of 29 references.
References · 12
- 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 208
- 2019Gaze360: Physically Unconstrained Gaze Estimation in the Wildcited by 174
- 2015Appearance-Based Gaze Estimation in the Wildcited by 153
- 2017MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimationcited by 118
- 2022GIMO: Gaze-Informed Human Motion Prediction in Contextcited by 81
- 2020In the Eye of the Beholder: Gaze and Actions in First Person Videocited by 69
- 2024Aria Everyday Activities Datasetcited by 39
- 2024Learning from Observer Gaze:Zero-Shot Attention Prediction Oriented by Human-Object Interaction Recognitioncited by 17
- 2025EgoM2P: Egocentric Multimodal Multitask Pretrainingcited by 9
- 2025Reading Recognition in the Wildcited by 6
- 2026Thinking with Gaze: Sequential Eye-Tracking as Visual Reasoning Supervision for Medical VLMscited by 2
- 2025Tokenization of Gaze Datacited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Real Eyes Realize Faster: Gaze Stability and Pupil Novelty for Efficient Egocentric Learning
- 2026EgoGazeLite: On-Device Egocentric Gaze Prediction for Token-Efficient Multimodal LLM Video Input
- 2026GazeFlow: From Human Gaze Behavior to Generative Egocentric Gaze Prediction
- 2026ARGaze: Autoregressive Transformers for Online Egocentric Gaze Estimation
- 2026Gaze-Regularized VLMs for Ego-Centric Behavior Understanding
- 2026EventPrune: Cascaded Event-Assisted Token Pruning for Efficient First-Person Dynamic Spatial Reasoning
- 2025Gaze-VLM:Bridging Gaze and VLMs through Attention Regularization for Egocentric Understanding
- 2026Diffusion models for eye-gaze trajectory generation using position and velocity representations