# Language Models Largely Exhibit Human-like Constituent Ordering Preferences

arXiv: https://arxiv.org/abs/2502.05670 · HTML: https://mlsanity.com/paper/2502.05670/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 53 references that are in the map.
- First posted on arXiv 8 Feb 2025.

## Builds on (most cited first, 11 of 11)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024)
- [Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies](https://mlsanity.com/paper/1611.01368/) (2016)
- [Targeted Syntactic Evaluation of Language Models](https://mlsanity.com/paper/1808.09031/) (2018)
- [What do RNN Language Models Learn about Filler-Gap Dependencies?](https://mlsanity.com/paper/1809.00042/) (2018)
- [minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models](https://mlsanity.com/paper/2203.13112/) (2022)
- [On the proper role of linguistically-oriented deep net analysis in linguistic theorizing](https://mlsanity.com/paper/2106.08694/) (2021)
- [RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency](https://mlsanity.com/paper/1809.01329/) (2018)
- [When a sentence does not introduce a discourse entity, Transformer-based models still sometimes refer to it](https://mlsanity.com/paper/2205.03472/) (2022)
- [Do RNNs learn human-like abstract word order preferences?](https://mlsanity.com/paper/1811.01866/) (2018)
- [What Syntactic Structures block Dependencies in RNN Language Models?](https://mlsanity.com/paper/1905.10431/) (2019)

## Built on by (most cited first, 1 of 1)

- [Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Models](https://mlsanity.com/paper/2503.20850/) (2025)

## Abstract (arXiv)

Though English sentences are typically inflexible vis-`a-vis word order, constituents often show far more variability in ordering. One prominent theory presents the notion that constituent ordering is directly correlated with constituent weight: a measure of the constituent's length or complexity. Such theories are interesting in the context of natural language processing (NLP), because while recent advances in NLP have led to significant gains in the performance of large language models (LLMs), much remains unclear about how these models process language, and how this compares to human language processing. In particular, the question remains whether LLMs display the same patterns with constituent movement, and may provide insights into existing theories on when and how the shift occurs in human language. We compare a variety of LLMs with diverse properties to evaluate broad LLM performance on four types of constituent movement: heavy NP shift, particle movement, dative alternation, and multiple PPs. Despite performing unexpectedly around particle movement, LLMs generally align with human preferences around constituent ordering.