mlsanityData to 14 Oct 2026

Investigating Syntactic Biases in Multilingual Transformers with RC Attachment Ambiguities in Italian and English

arXiv 2504.09886 ↗First posted 14 Apr 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper leverages past sentence processing studies to investigate whether monolingual and multilingual LLMs show human-like preferences when presented with examples of relative clause attachment ambiguities in Italian and English. Furthermore, we test whether these preferences can be modulated by lexical factors (the type of verb/noun in the matrix clause) which have been shown to be tied to subtle constraints on syntactic and semantic relations. Our results overall showcase how LLM behavior varies interestingly across models, but also general failings of these models in correctly capturing human-like preferences. In light of these results, we argue that RC attachment is the ideal benchmark for cross-linguistic investigations of LLMs' linguistic knowledge and biases.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 76 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

31 of 76 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  6. 2018Neural Network Acceptability Judgmentscited by 958
  7. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
  8. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 354
  9. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 320
  10. 2018Targeted Syntactic Evaluation of Language Modelscited by 242
  11. 2018Colorless green recurrent networks dream hierarchicallycited by 213
  12. 2019Assessing BERT's Syntactic Abilitiescited by 192
19 more
  1. 2019Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic Statecited by 138
  2. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 126
  3. 2020Syntactic Structure from Deep Learningcited by 99
  4. 2020UnNatural Language Inferencecited by 69
  5. 2022minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Modelscited by 58
  6. 2024Auxiliary task demands mask the capabilities of smaller language modelscited by 54
  7. 2024CausalGym: Benchmarking causal interpretability methods on linguistic taskscited by 46
  8. 2021Structural Persistence in Language Models: Priming as a Window into Abstract Language Representationscited by 44
  9. 2021Refining Targeted Syntactic Evaluation of Language Modelscited by 41
  10. 2022Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scalecited by 41
  11. 2021Language Models Use Monotonicity to Assess NPI Licensingcited by 31
  12. 2020Recurrent Neural Network Language Models Always Learn English-Like Relative Clause Attachmentcited by 18
  13. 2020GePpeTto Carves Italian into a Language Modelcited by 16
  14. 2024Incremental Comprehension of Garden-Path Sentences by Large Language Models: Semantic Interpretation, Syntactic Re-Analysis, and Attentioncited by 11
  15. 2021John praised Mary because he? Implicit Causality Bias and Its Interaction with Explicit Cues in LMscited by 5
  16. 2024Preuve de concept d'un bot vocal dialoguant en wolofcited by 3
  17. 2025Who Relies More on World Knowledge and Bias for Syntactic Ambiguity Resolution: Humans or LLMs?cited by 3
  18. 2022Sentence Ambiguity, Grammaticality and Complexity Probescited by 1
  19. 2025Implicit Causality-biases in humans and LLMs as a tool for benchmarking LLM discourse capabilitiescited by 1

Built on by

Not cited yet.