# Fine-Grained Prediction of Reading Comprehension from Eye Movements

arXiv: https://arxiv.org/abs/2410.04484 · HTML: https://mlsanity.com/paper/2410.04484/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 54 references that are in the map.
- First posted on arXiv 6 Oct 2024.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [Multimodal Learning with Transformers: A Survey](https://mlsanity.com/paper/2206.06488/) (2022)
- [On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines](https://mlsanity.com/paper/2006.04884/) (2020)
- [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://mlsanity.com/paper/2304.05613/) (2023)
- [Deep Eyedentification: Biometric Identification using Micro-Movements of the Eye](https://mlsanity.com/paper/1906.11889/) (2019)
- [Bridging Information-Seeking Human Gaze and Machine Reading Comprehension](https://mlsanity.com/paper/2009.14780/) (2020)
- [Eyettention: An Attention-based Dual-Sequence Model for Predicting Human Scanpaths during Reading](https://mlsanity.com/paper/2304.10784/) (2023)
- [STARC: Structured Annotations for Reading Comprehension](https://mlsanity.com/paper/2004.14797/) (2020)
- [Assessing Language Proficiency from Eye Movements in Reading](https://mlsanity.com/paper/1804.07329/) (2018)
- [Modeling Task Effects in Human Reading with Neural Network-based Attention](https://mlsanity.com/paper/1808.00054/) (2018)
- [SwissBERT: The Multilingual Language Model for Switzerland](https://mlsanity.com/paper/2303.13310/) (2023)
- [A Discriminative Model for Identifying Readers and Assessing Text Comprehension from Eye Movements](https://mlsanity.com/paper/1809.08031/) (2018)

## Built on by (most cited first, 9 of 9)

- [Decoding Reading Goals from Eye Movements](https://mlsanity.com/paper/2410.20779/) (2024)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [Controlling Reading Ease with Gaze-Guided Text Generation](https://mlsanity.com/paper/2601.17781/) (2026)
- [Integrating Cognitive Processing Signals into Language Models: A Review of Advances, Applications and Future Directions](https://mlsanity.com/paper/2504.06843/) (2025)
- [D'ej`a Vu? Decoding Repeated Reading from Eye Movements](https://mlsanity.com/paper/2502.11061/) (2025)
- [Decoding Open-Ended Information Seeking Goals from Eye Movements in Reading](https://mlsanity.com/paper/2505.02872/) (2025)
- [Eye-Tracking-while-Reading: A Living Survey of Datasets with Open Library Support](https://mlsanity.com/paper/2602.19598/) (2026)
- [LEXIC: Lightweight On-Device Decoding of Reading Comprehension from Eye Movements](https://mlsanity.com/paper/2607.08152/) (2026)
- [Language Proficiency Assessment from Eye Movements in Naturalistic Passage Reading](https://mlsanity.com/paper/2608.30583/) (2026)

## Abstract (arXiv)

Can human reading comprehension be assessed from eye movements in reading? In this work, we address this longstanding question using large-scale eyetracking data over textual materials that are geared towards behavioral analyses of reading comprehension. We focus on a fine-grained and largely unaddressed task of predicting reading comprehension from eye movements at the level of a single question over a passage. We tackle this task using three new multimodal language models, as well as a battery of prior models from the literature. We evaluate the models' ability to generalize to new textual items, new participants, and the combination of both, in two different reading regimes, ordinary reading and information seeking. The evaluations suggest that although the task is highly challenging, eye movements contain useful signals for fine-grained prediction of reading comprehension. Code and data will be made publicly available.