# Do RNNs learn human-like abstract word order preferences?

arXiv: https://arxiv.org/abs/1811.01866 · HTML: https://mlsanity.com/paper/1811.01866/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2018.

## Built on by (most cited first, 10 of 10)

- [The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values](https://mlsanity.com/paper/2310.07629/) (2023)
- [Preregistering NLP Research](https://mlsanity.com/paper/2103.06944/) (2021)
- [Emergent Word Order Universals from Cognitively-Motivated Language Models](https://mlsanity.com/paper/2402.12363/) (2024)
- [A multilabel approach to morphosyntactic probing](https://mlsanity.com/paper/2104.08464/) (2021)
- [Do Grammatical Error Correction Models Realize Grammatical Generalization?](https://mlsanity.com/paper/2106.03031/) (2021)
- [Evaluating Models of Robust Word Recognition with Serial Reproduction](https://mlsanity.com/paper/2101.09788/) (2021)
- [Lil-Bevo: Explorations of Strategies for Training Language Models in More Humanlike Ways](https://mlsanity.com/paper/2310.17591/) (2023)
- [Language Models Largely Exhibit Human-like Constituent Ordering Preferences](https://mlsanity.com/paper/2502.05670/) (2025)
- [Black Big Boxes: Tracing Adjective Order Preferences in Large Language Models](https://mlsanity.com/paper/2407.02136/) (2024)
- [Factors Influencing the Emergence of Dependency Length Minimization in Neural Agent Simulations](https://mlsanity.com/paper/2609.06025/) (2026)

## Abstract (arXiv)

RNN language models have achieved state-of-the-art results on various tasks, but what exactly they are representing about syntax is as yet unclear. Here we investigate whether RNN language models learn humanlike word order preferences in syntactic alternations. We collect language model surprisal scores for controlled sentence stimuli exhibiting major syntactic alternations in English: heavy NP shift, particle shift, the dative alternation, and the genitive alternation. We show that RNN language models reproduce human preferences in these alternations based on NP length, animacy, and definiteness. We collect human acceptability ratings for our stimuli, in the first acceptability judgment experiment directly manipulating the predictors of syntactic alternations. We show that the RNNs' performance is similar to the human acceptability ratings and is not matched by an n-gram baseline model. Our results show that RNNs learn the abstract features of weight, animacy, and definiteness which underlie soft constraints on syntactic alternations.