mlsanityData to 7 Oct 2026

Cross-Lingual Transfer of Cognitive Processing Complexity

arXiv 2302.12695 ↗First posted 24 Feb 2023cs.CL cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

When humans read a text, their eye movements are influenced by the structural complexity of the input sentences. This cognitive phenomenon holds across languages and recent studies indicate that multilingual language models utilize structural similarities between languages to facilitate cross-lingual transfer. We use sentence-level eye-tracking patterns as a cognitive indicator for structural complexity and show that the multilingual model XLM-RoBERTa can successfully predict varied patterns for 13 typologically diverse languages, despite being fine-tuned only on English data. We quantify the sensitivity of the model to structural complexity and distinguish a range of complexity characteristics. Our results indicate that the model develops a meaningful bias towards sentence length but also integrates cross-lingual differences. We conduct a control experiment with randomized word order and find that the model seems to additionally capture more complex structural information.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 21 of its 52 references that are in the map. First posted on arXiv 24 Feb 2023.

Builds on

21 of 52 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  3. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  4. 2019How multilingual is Multilingual BERT?cited by 642
  5. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  6. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  7. 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
  8. 2020On the Language Neutrality of Pre-trained Multilingual Representationscited by 82
  9. 2021When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfercited by 41
  10. 2020It's not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERTcited by 30
  11. 2021Multilingual Language Models Predict Human Reading Behaviorcited by 29
  12. 2020Probing Multilingual BERT for Genetic and Typological Signalscited by 26
9 more
  1. 2019What do Language Representations Really Represent?cited by 17
  2. 2022Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structurecited by 17
  3. 2019Semantic Drift in Multilingual Representationscited by 13
  4. 2021Syntax-augmented Multilingual BERT for Cross-lingual Transfercited by 11
  5. 2020A Study of Cross-Lingual Ability and Language-specific Information in Multilingual BERTcited by 8
  6. 2021On the Language-specificity of Multilingual BERT and the Impact of Fine-tuningcited by 8
  7. 2022Measuring the Impact of (Psycho-)Linguistic and Readability Features and Their Spill Over Effects on the Prediction of Eye Movement Patternscited by 8
  8. 2022Every word counts: A multilingual analysis of individual human alignment with model attentioncited by 4
  9. 2022Zero Shot Crosslingual Eye-Tracking Data Prediction using Multilingual Transformer Modelscited by 3

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1.

2 papers.

  1. 2023Psychometric Predictive Power of Large Language Modelscited by 4
  2. 2024Describing Images textitFast and Slow: Quantifying and Predicting the Variation in Human Signals during Visuo-Linguistic Processescited by 1