# For GPT-4 as with Humans: Information Structure Predicts Acceptability of Long-Distance Dependencies

arXiv: https://arxiv.org/abs/2505.09005 · HTML: https://mlsanity.com/paper/2505.09005/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 42 references that are in the map.
- First posted on arXiv 13 May 2025.

## Builds on (most cited first, 9 of 9)

- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition](https://mlsanity.com/paper/2004.03066/) (2020)
- [PUB: A Pragmatics Understanding Benchmark for Assessing LLMs' Pragmatics Capabilities](https://mlsanity.com/paper/2401.07078/) (2024)
- [Language models align with human judgments on key grammatical constructions](https://mlsanity.com/paper/2402.01676/) (2024)
- [Testing AI on language comprehension tasks reveals insensitivity to underlying meaning](https://mlsanity.com/paper/2302.12313/) (2023)
- [Prompting is not a substitute for probability measurements in large language models](https://mlsanity.com/paper/2305.13264/) (2023)
- [Do Language Models' Words Refer?](https://mlsanity.com/paper/2308.05576/) (2023)
- [Generalizations across filler-gap dependencies in neural language models](https://mlsanity.com/paper/2410.18225/) (2024)
- [Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds](https://mlsanity.com/paper/2305.14785/) (2023)

## Built on by (most cited first, 1 of 1)

- [When Discourse Pressures Conflict: Information Structure in Vision-Language Model Outputs](https://mlsanity.com/paper/2605.28346/) (2026)

## Abstract (arXiv)

It remains debated how well any LM understands natural language or generates reliable metalinguistic judgments. Moreover, relatively little work has demonstrated that LMs can represent and respect subtle relationships between form and function proposed by linguists. We here focus on a particular such relationship established in recent work: English speakers' judgments about the information structure of canonical sentences predicts independently collected acceptability ratings on corresponding 'long distance dependency' [LDD] constructions, across a wide array of base constructions and multiple types of LDDs. To determine whether any LM captures this relationship, we probe GPT-4 on the same tasks used with humans and new extensions.Results reveal reliable metalinguistic skill on the information structure and acceptability tasks, replicating a striking interaction between the two, despite the zero-shot, explicit nature of the tasks, and little to no chance of contamination [Studies 1a, 1b]. Study 2 manipulates the information structure of base sentences and confirms a causal relationship: increasing the prominence of a constituent in a context sentence increases the subsequent acceptability ratings on an LDD construction. The findings suggest a tight relationship between natural and GPT-4 generated English, and between information structure and syntax, which begs for further exploration.