# The Language of Looking: Motion Tokens Reveal Domain-Dependent Local Structure in Gaze Sequences

arXiv: https://arxiv.org/abs/2610.16565 · HTML: https://mlsanity.com/paper/2610.16565/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 43 references that are in the map.
- First posted on arXiv 12 Oct 2026.

## Builds on (most cited first, 10 of 10)

- [Gaze360: Physically Unconstrained Gaze Estimation in the Wild](https://mlsanity.com/paper/1910.10088/) (2019)
- [ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotation](https://mlsanity.com/paper/1912.00903/) (2019)
- [Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attention](https://mlsanity.com/paper/2303.15274/) (2023)
- [Eyettention: An Attention-based Dual-Sequence Model for Predicting Human Scanpaths during Reading](https://mlsanity.com/paper/2304.10784/) (2023)
- [ScanDL: A Diffusion Model for Generating Synthetic Scanpaths on Texts](https://mlsanity.com/paper/2310.15587/) (2023)
- [On the Proper Treatment of Tokenization in Psycholinguistics](https://mlsanity.com/paper/2410.02691/) (2024)
- [OAT: Object-Level Attention Transformer for Gaze Scanpath Prediction](https://mlsanity.com/paper/2407.13335/) (2024)
- [Decoding Reading Goals from Eye Movements](https://mlsanity.com/paper/2410.20779/) (2024)
- [Thinking with Gaze: Sequential Eye-Tracking as Visual Reasoning Supervision for Medical VLMs](https://mlsanity.com/paper/2603.06697/) (2026)
- [Motion-Based Tokenization for Cross-Dataset Egocentric Gaze Modeling](https://mlsanity.com/paper/2608.22926/) (2026)

## Abstract (arXiv)

Human gaze is the primary high-bandwidth interface through which we parse the semantics of our environment. While often modeled as a sequence, the degree to which gaze exhibits transferable local structure across domains remains underexplored. Using motion tokenization as a diagnostic lens across four diverse datasets, we investigate the transferability of local sequence structure. We demonstrate that while gaze can share a reusable motion-token vocabulary, identical motion tokens are organized into different local transition patterns depending on the dataset and task context. Our results show the limitations of naive data-pooling under low-semantic motion-token representations and provide a framework for evaluating behavioral sequences through information-theoretic diagnostics.