# Investigating Syntactic Biases in Multilingual Transformers with RC Attachment Ambiguities in Italian and English

arXiv: https://arxiv.org/abs/2504.09886 · HTML: https://mlsanity.com/paper/2504.09886/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 76 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies](https://mlsanity.com/paper/1611.01368/) (2016)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [Targeted Syntactic Evaluation of Language Models](https://mlsanity.com/paper/1808.09031/) (2018)
- [Colorless green recurrent networks dream hierarchically](https://mlsanity.com/paper/1803.11138/) (2018)
- [Assessing BERT's Syntactic Abilities](https://mlsanity.com/paper/1901.05287/) (2019)
- [Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State](https://mlsanity.com/paper/1903.03260/) (2019)
- [We're Afraid Language Models Aren't Modeling Ambiguity](https://mlsanity.com/paper/2304.14399/) (2023)
- [Syntactic Structure from Deep Learning](https://mlsanity.com/paper/2004.10827/) (2020)
- [UnNatural Language Inference](https://mlsanity.com/paper/2101.00010/) (2020)
- [minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models](https://mlsanity.com/paper/2203.13112/) (2022)
- [Auxiliary task demands mask the capabilities of smaller language models](https://mlsanity.com/paper/2404.02418/) (2024)
- [CausalGym: Benchmarking causal interpretability methods on linguistic tasks](https://mlsanity.com/paper/2402.12560/) (2024)
- [Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations](https://mlsanity.com/paper/2109.14989/) (2021)
- [Refining Targeted Syntactic Evaluation of Language Models](https://mlsanity.com/paper/2104.09635/) (2021)
- [Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale](https://mlsanity.com/paper/2203.00633/) (2022)
- [Language Models Use Monotonicity to Assess NPI Licensing](https://mlsanity.com/paper/2105.13818/) (2021)
- [Recurrent Neural Network Language Models Always Learn English-Like Relative Clause Attachment](https://mlsanity.com/paper/2005.00165/) (2020)
- [GePpeTto Carves Italian into a Language Model](https://mlsanity.com/paper/2004.14253/) (2020)
- [Incremental Comprehension of Garden-Path Sentences by Large Language Models: Semantic Interpretation, Syntactic Re-Analysis, and Attention](https://mlsanity.com/paper/2405.16042/) (2024)
- [John praised Mary because he? Implicit Causality Bias and Its Interaction with Explicit Cues in LMs](https://mlsanity.com/paper/2106.01060/) (2021)
- [Preuve de concept d'un bot vocal dialoguant en wolof](https://mlsanity.com/paper/2404.02009/) (2024)
- [Who Relies More on World Knowledge and Bias for Syntactic Ambiguity Resolution: Humans or LLMs?](https://mlsanity.com/paper/2503.10838/) (2025)
- [Sentence Ambiguity, Grammaticality and Complexity Probes](https://mlsanity.com/paper/2210.06928/) (2022)

## Abstract (arXiv)

This paper leverages past sentence processing studies to investigate whether monolingual and multilingual LLMs show human-like preferences when presented with examples of relative clause attachment ambiguities in Italian and English. Furthermore, we test whether these preferences can be modulated by lexical factors (the type of verb/noun in the matrix clause) which have been shown to be tied to subtle constraints on syntactic and semantic relations. Our results overall showcase how LLM behavior varies interestingly across models, but also general failings of these models in correctly capturing human-like preferences. In light of these results, we argue that RC attachment is the ideal benchmark for cross-linguistic investigations of LLMs' linguistic knowledge and biases.