mlsanityData to 13 Oct 2026

Multilingual Language Models Predict Human Reading Behavior

arXiv 2104.05433 ↗First posted 12 Apr 2021cs.CLcited by 29

Not on the map. Why?

Abstract (arXiv)

We analyze if large language models are able to predict patterns of human reading behavior. We compare the performance of language-specific and multilingual pretrained transformer models to predict reading time measures reflecting natural human sentence processing on Dutch, English, German, and Russian texts. This results in accurate models of human reading behavior, which indicates that transformer models implicitly encode relative importance in language in a way that is comparable to human processing mechanisms. We find that BERT and XLM models successfully predict a range of eye tracking features. In a series of experiments, we analyze the cross-domain and cross-language abilities of these models and show how they reflect human sentence processing.

Key facts. Cited by 29 papers in the map (data through 13 Oct 2026). Builds on 30 of its 66 references that are in the map. First posted on arXiv 12 Apr 2021.

Builds on

30 of 66 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,106
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,916
  4. 2019Cross-lingual Language Model Pretrainingcited by 1,089
  5. 2019Attention is not Explanationcited by 922
  6. 2019How multilingual is Multilingual BERT?cited by 643
  7. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  8. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  9. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 222
  10. 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
  11. 2019Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)cited by 128
  12. 2020The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?cited by 122
18 more
  1. 2019Multilingual is not enough: BERT for Finnishcited by 118
  2. 2020On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behaviorcited by 118
  3. 2019Adaptation of Deep Bidirectional Multilingual Transformers for Russian Languagecited by 109
  4. 2019BERTje: A Dutch BERT Modelcited by 109
  5. 2019ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotationcited by 65
  6. 2020How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Maskingcited by 63
  7. 2020Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attentioncited by 45
  8. 2020Multi-SimLex: A Large-Scale Evaluation of Multilingual and Cross-Lingual Lexical Semantic Similaritycited by 38
  9. 2019Blackbox meets blackbox: Representational Similarity and Stability Analysis of Neural Language Models and Brainscited by 37
  10. 2019Entity Recognition at First Sight: Improving NER with Eye Movement Informationcited by 26
  11. 2016Improving sentence compression by learning to predict gazecited by 24
  12. 2020Probabilistic Predictions of People Perusing: Evaluating Metrics of Language Model Performance for Psycholinguistic Modelingcited by 22
  13. 2020Investigating representations of verb bias in neural language modelscited by 22
  14. 2020Bridging Information-Seeking Human Gaze and Machine Reading Comprehensioncited by 20
  15. 2019Advancing NLP with Cognitive Language Processing Signalscited by 18
  16. 2020How well does surprisal explain N400 amplitude under different experimental conditions?cited by 18
  17. 2020Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gazecited by 16
  18. 2016Modeling Human Reading with Neural Attentioncited by 10

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 7; 2024: 7; 2025: 3; 2026: 4.

29 papers.

Sort:

By area:

  1. 2023Testing the Predictions of Surprisal Theory in 11 Languagescited by 76
  2. 2025On the Emergence of Position Bias in Transformerscited by 58
  3. 2021The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Failcited by 38
  4. 2021Relative Importance in Sentence Processingcited by 22
  5. 2022Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?cited by 20
  6. 2023Eyettention: An Attention-based Dual-Sequence Model for Predicting Human Scanpaths during Readingcited by 18
  7. 2023Synthesizing Human Gaze Feedback for Improved NLP Performancecited by 16
  8. 2023Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Modelscited by 16
  9. 2023ScanDL: A Diffusion Model for Generating Synthetic Scanpaths on Textscited by 13
  10. 2024From Babble to Words: Pre-Training Language Models on Continuous Streams of Phonemescited by 11
  11. 2026The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices?cited by 10
  12. 2022Measuring the Impact of (Psycho-)Linguistic and Readability Features and Their Spill Over Effects on the Prediction of Eye Movement Patternscited by 8
17 more
  1. 2024Large Language Models for Psycholinguistic Plausibility Pretestingcited by 5
  2. 2024PoTeC: A German Naturalistic Eye-tracking-while-reading Corpuscited by 5
  3. 2024EMTeC: A Corpus of Eye Movements on Machine-Generated Textscited by 5
  4. 2022Every word counts: A multilingual analysis of individual human alignment with model attentioncited by 4
  5. 2025OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responsescited by 3
  6. 2023Cross-Lingual Transfer of Cognitive Processing Complexitycited by 2
  7. 2024Describing Images textitFast and Slow: Quantifying and Predicting the Variation in Human Signals during Visuo-Linguistic Processescited by 1
  8. 2024Reverse-Engineering the Readercited by 1
  9. 2026Plausibility as Commonsense Reasoning: Humans Succeed, Large Language Models Do notcited by 1
  10. 2022NU HLT at CMCL 2022 Shared Task: Multilingual and Crosslingual Prediction of Human Reading Behavior in Universal Language Spacecited by 0
  11. 2022Team 'UFAL at CMCL 2022 Shared Task: Figuring out the correct recipe for predicting Eye-Tracking features using Pretrained Language Modelscited by 0
  12. 2022Text Difficulty Study: Do machines behave the same as humans regarding text difficulty?cited by 0
  13. 2023Revising with a Backward Glance: Regressions and Skips during Reading as Cognitive Signals for Revision Policies in Incremental Processingcited by 0
  14. 2024Evaluating Webcam-based Gaze Data as an Alternative for Human Rationale Annotationscited by 0
  15. 2025Decoding Open-Ended Information Seeking Goals from Eye Movements in Readingcited by 0
  16. 2026Eyettention II: A Dual-Sequence Architecture for Modeling Fixation Location, Within-Word Landing Position, and Fixation Duration in Readingcited by 0
  17. 2026Do Cantonese-Adapted Language Models Better Predict Cantonese Reading? A Cross-Model Eye-Tracking Evaluationcited by 0