# A Comparative Study on Textual Saliency of Styles from Eye Tracking, Annotations, and Language Models

arXiv: https://arxiv.org/abs/2212.09873 · HTML: https://mlsanity.com/paper/2212.09873/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 42 references that are in the map.
- First posted on arXiv 19 Dec 2022.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior](https://mlsanity.com/paper/2006.01912/) (2020)
- [A Computational Approach to Politeness with Application to Social Factors](https://mlsanity.com/paper/1306.6078/) (2013)
- [Meta-Learning to Compositionally Generalize](https://mlsanity.com/paper/2106.04252/) (2021)
- [Lower Perplexity is Not Always Human-Like](https://mlsanity.com/paper/2106.01229/) (2021)
- [Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention](https://mlsanity.com/paper/2010.07891/) (2020)
- [Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension](https://mlsanity.com/paper/2010.06396/) (2020)
- [Does BERT Learn as Humans Perceive? Understanding Linguistic Styles through Lexica](https://mlsanity.com/paper/2109.02738/) (2021)
- [Bridging Information-Seeking Human Gaze and Machine Reading Comprehension](https://mlsanity.com/paper/2009.14780/) (2020)
- [Advancing NLP with Cognitive Language Processing Signals](https://mlsanity.com/paper/1904.02682/) (2019)
- [A Survey on Using Gaze Behaviour for Natural Language Processing](https://mlsanity.com/paper/2112.15471/) (2021)
- [Harnessing Cognitive Features for Sarcasm Detection](https://mlsanity.com/paper/1701.05574/) (2017)
- [StyLEx: Explaining Style Using Human Lexical Annotations](https://mlsanity.com/paper/2210.07469/) (2022)

## Built on by (most cited first, 2 of 2)

- [Learning Explainable Dense Reward Shapes via Bayesian Optimization](https://mlsanity.com/paper/2504.16272/) (2025)
- [Reasoning Beyond Literal: Cross-style Multimodal Reasoning for Figurative Language Understanding](https://mlsanity.com/paper/2601.17197/) (2026)

## Abstract (arXiv)

There is growing interest in incorporating eye-tracking data and other implicit measures of human language processing into natural language processing (NLP) pipelines. The data from human language processing contain unique insight into human linguistic understanding that could be exploited by language models. However, many unanswered questions remain about the nature of this data and how it can best be utilized in downstream NLP tasks. In this paper, we present eyeStyliency, an eye-tracking dataset for human processing of stylistic text (e.g., politeness). We develop a variety of methods to derive style saliency scores over text using the collected eye dataset. We further investigate how this saliency data compares to both human annotation methods and model-based interpretability metrics. We find that while eye-tracking data is unique, it also intersects with both human annotations and model-based importance scores, providing a possible bridge between human- and machine-based perspectives. We propose utilizing this type of data to evaluate the cognitive plausibility of models that interpret style. Our eye-tracking data and processing code are publicly available.