# Eyes are the Windows to the Soul: Predicting the Rating of Text Quality Using Gaze Behaviour

arXiv: https://arxiv.org/abs/1810.04839 · HTML: https://mlsanity.com/paper/1810.04839/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2018.

## Built on by (most cited first, 9 of 9)

- [Many Hands Make Light Work: Using Essay Traits to Automatically Score Essays](https://mlsanity.com/paper/2102.00781/) (2021)
- [Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models](https://mlsanity.com/paper/2410.01532/) (2024)
- [A Survey on Using Gaze Behaviour for Natural Language Processing](https://mlsanity.com/paper/2112.15471/) (2021)
- [PoTeC: A German Naturalistic Eye-tracking-while-reading Corpus](https://mlsanity.com/paper/2403.00506/) (2024)
- [EMTeC: A Corpus of Eye Movements on Machine-Generated Texts](https://mlsanity.com/paper/2408.04289/) (2024)
- [Progressive Document-level Text Simplification via Large Language Models](https://mlsanity.com/paper/2501.03857/) (2025)
- [OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responses](https://mlsanity.com/paper/2503.10927/) (2025)
- [Cognitively Aided Zero-Shot Automatic Essay Grading](https://mlsanity.com/paper/2102.11258/) (2021)
- [Eye-Tracking-while-Reading: A Living Survey of Datasets with Open Library Support](https://mlsanity.com/paper/2602.19598/) (2026)

## Abstract (arXiv)

Predicting a reader's rating of text quality is a challenging task that involves estimating different subjective aspects of the text, like structure, clarity, etc. Such subjective aspects are better handled using cognitive information. One such source of cognitive information is gaze behaviour. In this paper, we show that gaze behaviour does indeed help in effectively predicting the rating of text quality. To do this, we first model text quality as a function of three properties - organization, coherence and cohesion. Then, we demonstrate how capturing gaze behaviour helps in predicting each of these properties, and hence the overall quality, by reporting improvements obtained by adding gaze features to traditional textual features for score prediction. We also hypothesize that if a reader has fully understood the text, the corresponding gaze behaviour would give a better indication of the assigned rating, as opposed to partial understanding. Our experiments validate this hypothesis by showing greater agreement between the given rating and the predicted rating when the reader has a full understanding of the text.