# Motion-Based Tokenization for Cross-Dataset Egocentric Gaze Modeling

arXiv: https://arxiv.org/abs/2608.22926 · HTML: https://mlsanity.com/paper/2608.22926/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 29 references that are in the map.
- First posted on arXiv 24 Aug 2026.

## Builds on (most cited first, 12 of 12)

- [Project Aria: A New Tool for Egocentric Multi-Modal AI Research](https://mlsanity.com/paper/2308.13561/) (2023)
- [Gaze360: Physically Unconstrained Gaze Estimation in the Wild](https://mlsanity.com/paper/1910.10088/) (2019)
- [Appearance-Based Gaze Estimation in the Wild](https://mlsanity.com/paper/1504.02863/) (2015)
- [MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation](https://mlsanity.com/paper/1711.09017/) (2017)
- [GIMO: Gaze-Informed Human Motion Prediction in Context](https://mlsanity.com/paper/2204.09443/) (2022)
- [In the Eye of the Beholder: Gaze and Actions in First Person Video](https://mlsanity.com/paper/2006.00626/) (2020)
- [Aria Everyday Activities Dataset](https://mlsanity.com/paper/2402.13349/) (2024)
- [Learning from Observer Gaze:Zero-Shot Attention Prediction Oriented by Human-Object Interaction Recognition](https://mlsanity.com/paper/2405.09931/) (2024)
- [EgoM2P: Egocentric Multimodal Multitask Pretraining](https://mlsanity.com/paper/2506.07886/) (2025)
- [Reading Recognition in the Wild](https://mlsanity.com/paper/2505.24848/) (2025)
- [Thinking with Gaze: Sequential Eye-Tracking as Visual Reasoning Supervision for Medical VLMs](https://mlsanity.com/paper/2603.06697/) (2026)
- [Tokenization of Gaze Data](https://mlsanity.com/paper/2503.22145/) (2025)

## Built on by (most cited first, 1 of 1)

- [The Language of Looking: Motion Tokens Reveal Domain-Dependent Local Structure in Gaze Sequences](https://mlsanity.com/paper/2610.16565/) (2026)

## Abstract (arXiv)

Gaze is increasingly used as an input signal for vision and multimodal models, yet no consensus exists on how to represent it across datasets. Raw traces preserve detail but are noisy and device-dependent, while coarse event labels are easy to model but can discard local motion structure. We formulate event-aligned, fixed-horizon angular displacement as an interpretable, event-conditioned motion vocabulary and compare it with event-only, spatial, absolute-angle, learned vector-quantized, and continuous representations. To assess transfer alongside target predictability and token collapse, our evaluation combines next-token prediction with target-domain regret, low-order target references, paired bootstrap, order sensitivity, motif overlap, and frozen structural probes. In an event-aligned headset benchmark, angular-motion tokens have lower target-domain regret than frozen-codebook VQ tokens in one transfer direction, while the reverse direction is inconclusive. The probes reveal complementary representation properties, and event-only tokens show that low perplexity can retain little motion information. On a third egocentric dataset, a matched comparison of I-VT, native, and frame-span interfaces shows that event construction materially changes transfer: native events have the lowest regret into EGTEA, while frame-span events have zero motif overlap and fail severely as a source. Motion-based tokenization therefore provides a compact representation for event-aligned egocentric gaze streams, while the evaluation identifies how target predictability and event construction shape cross-dataset conclusions.