# Do language models make human-like predictions about the coreferents of Italian anaphoric zero pronouns?

arXiv: https://arxiv.org/abs/2208.14554 · HTML: https://mlsanity.com/paper/2208.14554/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 87 references that are in the map.
- First posted on arXiv 30 Aug 2022.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [Few-shot Learning with Multilingual Language Models](https://mlsanity.com/paper/2112.10668/) (2021)
- [WinoGrande: An Adversarial Winograd Schema Challenge at Scale](https://mlsanity.com/paper/1907.10641/) (2019)
- [Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little](https://mlsanity.com/paper/2104.06644/) (2021)
- [Transformer Language Models without Positional Encodings Still Learn Positional Information](https://mlsanity.com/paper/2203.16634/) (2022)
- [As Good as New. How to Successfully Recycle English GPT-2 to Make Models for Other Languages](https://mlsanity.com/paper/2012.05628/) (2020)
- [Human Sentence Processing: Recurrence or Attention?](https://mlsanity.com/paper/2005.09471/) (2020)
- [BERT & Family Eat Word Salad: Experiments with Text Understanding](https://mlsanity.com/paper/2101.03453/) (2021)
- [On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment](https://mlsanity.com/paper/2010.03017/) (2020)
- [How well does surprisal explain N400 amplitude under different experimental conditions?](https://mlsanity.com/paper/2010.04844/) (2020)
- [GePpeTto Carves Italian into a Language Model](https://mlsanity.com/paper/2004.14253/) (2020)
- [An Empirical Study of Contextual Data Augmentation for Japanese Zero Anaphora Resolution](https://mlsanity.com/paper/2011.00948/) (2020)
- [Pseudo Zero Pronoun Resolution Improves Zero Anaphora Resolution](https://mlsanity.com/paper/2104.07425/) (2021)
- [Transformer-GCRF: Recovering Chinese Dropped Pronouns with General Conditional Random Fields](https://mlsanity.com/paper/2010.03224/) (2020)

## Built on by (most cited first, 1 of 1)

- [To Drop or Not to Drop? Predicting Argument Ellipsis Judgments: A Case Study in Japanese](https://mlsanity.com/paper/2404.11315/) (2024)

## Abstract (arXiv)

Some languages allow arguments to be omitted in certain contexts. Yet human language comprehenders reliably infer the intended referents of these zero pronouns, in part because they construct expectations about which referents are more likely. We ask whether Neural Language Models also extract the same expectations. We test whether 12 contemporary language models display expectations that reflect human behavior when exposed to sentences with zero pronouns from five behavioral experiments conducted in Italian by Carminati (2005). We find that three models - XGLM 2.9B, 4.5B, and 7.5B - capture the human behavior from all the experiments, with others successfully modeling some of the results. This result suggests that human expectations about coreference can be derived from exposure to language, and also indicates features of language models that allow them to better reflect human behavior.