# Attention-aware semantic relevance predicting Chinese sentence reading

arXiv: https://arxiv.org/abs/2403.18542 · HTML: https://mlsanity.com/paper/2403.18542/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 112 references that are in the map.
- First posted on arXiv 27 Mar 2024.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [mGPT: Few-Shot Learners Go Multilingual](https://mlsanity.com/paper/2204.07580/) (2022)
- [CogniVal: A Framework for Cognitive Word Embedding Evaluation](https://mlsanity.com/paper/1909.09001/) (2019)
- [Pre-Training with Whole Word Masking for Chinese BERT](https://mlsanity.com/paper/1906.08101/) (2019)

## Built on by (most cited first, 4 of 4)

- [Contextual Semantic Relevance Tracks fMRI BOLD Responses During Naturalistic Speech Comprehension](https://mlsanity.com/paper/2607.15856/) (2026)
- [The Roles of Contextual Semantic Relevance Metrics in Human Visual Processing](https://mlsanity.com/paper/2410.09921/) (2024)
- [Sequential Contextual Fit Predicts Human Behavioural and Neural Dynamics Across Domains](https://mlsanity.com/paper/2609.20179/) (2026)
- [A Systematic Analysis of the Predictive Power of LM Surprisal in Reading Chinese](https://mlsanity.com/paper/2610.04898/) (2026)

## Abstract (arXiv)

In recent years, several influential computational models and metrics have been proposed to predict how humans comprehend and process sentence. One particularly promising approach is contextual semantic similarity. Inspired by the attention algorithm in Transformer and human memory mechanisms, this study proposes an ``attention-aware'' approach for computing contextual semantic relevance. This new approach takes into account the different contributions of contextual parts and the expectation effect, allowing it to incorporate contextual information fully. The attention-aware approach also facilitates the simulation of existing reading models and evaluate them. The resulting ``attention-aware'' metrics of semantic relevance can more accurately predict fixation durations in Chinese reading tasks recorded in an eye-tracking corpus than those calculated by existing approaches. The study's findings further provide strong support for the presence of semantic preview benefits in Chinese naturalistic reading. Furthermore, the attention-aware metrics of semantic relevance, being memory-based, possess high interpretability from both linguistic and cognitive standpoints, making them a valuable computational tool for modeling eye-movements in reading and further gaining insight into the process of language comprehension. Our approach underscores the potential of these metrics to advance our comprehension of how humans understand and process language, ultimately leading to a better understanding of language comprehension and processing.