# Human Attention during Goal-directed Reading Comprehension Relies on Task Optimization

arXiv: https://arxiv.org/abs/2107.05799 · HTML: https://mlsanity.com/paper/2107.05799/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 79 references that are in the map.
- First posted on arXiv 13 Jul 2021.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019)
- [A Structured Self-attentive Sentence Embedding](https://mlsanity.com/paper/1703.03130/) (2017)
- [Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?](https://mlsanity.com/paper/1606.03556/) (2016)
- [A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task](https://mlsanity.com/paper/1606.02858/) (2016)
- [Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension](https://mlsanity.com/paper/2010.06396/) (2020)
- [Dual Co-Matching Network for Multi-choice Reading Comprehension](https://mlsanity.com/paper/1901.09381/) (2019)
- [Bridging Information-Seeking Human Gaze and Machine Reading Comprehension](https://mlsanity.com/paper/2009.14780/) (2020)
- [Option Comparison Network for Multiple-choice Reading Comprehension](https://mlsanity.com/paper/1903.03033/) (2019)
- [Modeling Task Effects in Human Reading with Neural Network-based Attention](https://mlsanity.com/paper/1808.00054/) (2018)
- [PALRACE: Reading Comprehension Dataset with Human Data and Labeled Rationales](https://mlsanity.com/paper/2106.12373/) (2021)

## Built on by (most cited first, 3 of 3)

- [Computational Models to Study Language Processing in the Human Brain: A Survey](https://mlsanity.com/paper/2403.13368/) (2024)
- [A recurrent vision transformer shows signatures of primate visual attention](https://mlsanity.com/paper/2502.10955/) (2025)
- [Difficulty-Controllable Cloze Question Distractor Generation](https://mlsanity.com/paper/2511.01526/) (2025)

## Abstract (arXiv)

The computational principles underlying attention allocation in complex goal-directed tasks remain elusive. Goal-directed reading, i.e., reading a passage to answer a question in mind, is a common real-world task that strongly engages attention. Here, we investigate what computational models can explain attention distribution in this complex task. We show that the reading time on each word is predicted by the attention weights in transformer-based deep neural networks (DNNs) optimized to perform the same reading task. Eye-tracking further reveals that readers separately attend to basic text features and question-relevant information during first-pass reading and rereading, respectively. Similarly, text features and question relevance separately modulate attention weights in shallow and deep DNN layers. Furthermore, when readers scan a passage without a question in mind, their reading time is predicted by DNNs optimized for a word prediction task. Therefore, attention during real-world reading can be interpreted as the consequence of task optimization.