# Learning Sentence Representation with Guidance of Human Attention

arXiv: https://arxiv.org/abs/1609.09189 · HTML: https://mlsanity.com/paper/1609.09189/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Sep 2016.

## Built on by (most cited first, 2 of 2)

- [Synthesizing Human Gaze Feedback for Improved NLP Performance](https://mlsanity.com/paper/2302.05721/) (2023)
- [Language Cognition and Language Computation -- Human and Machine Language Understanding](https://mlsanity.com/paper/2301.04788/) (2023)

## Abstract (arXiv)

Recently, much progress has been made in learning general-purpose sentence representations that can be used across domains. However, most of the existing models typically treat each word in a sentence equally. In contrast, extensive studies have proven that human read sentences efficiently by making a sequence of fixation and saccades. This motivates us to improve sentence representations by assigning different weights to the vectors of the component words, which can be treated as an attention mechanism on single sentences. To that end, we propose two novel attention models, in which the attention weights are derived using significant predictors of human reading time, i.e., Surprisal, POS tags and CCG supertags. The extensive experiments demonstrate that the proposed methods significantly improve upon the state-of-the-art sentence representation models.