# Investigating representations of verb bias in neural language models

arXiv: https://arxiv.org/abs/2010.02375 · HTML: https://mlsanity.com/paper/2010.02375/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2020.

## Built on by (most cited first, 22 of 22)

- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [Multilingual Language Models Predict Human Reading Behavior](https://mlsanity.com/paper/2104.05433/) (2021)
- [A Discerning Several Thousand Judgments: GPT-3 Rates the Article + Adjective + Numeral + Noun Construction](https://mlsanity.com/paper/2301.12564/) (2023)
- [Do Language Models Exhibit Human-like Structural Priming Effects?](https://mlsanity.com/paper/2406.04847/) (2024)
- [Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Models](https://mlsanity.com/paper/2503.20850/) (2025)
- [How Abstract Is Linguistic Generalization in Large Language Models? Experiments with Argument Structure](https://mlsanity.com/paper/2311.04900/) (2023)
- [Probing BERT's priors with serial reproduction chains](https://mlsanity.com/paper/2202.12226/) (2022)
- [Linguistic Generalizations are not Rules: Impacts on Evaluation of LMs](https://mlsanity.com/paper/2502.13195/) (2025)
- [A systematic framework for generating novel experimental hypotheses from language models](https://mlsanity.com/paper/2408.05086/) (2024)
- [Constructions are Revealed in Word Distributions](https://mlsanity.com/paper/2503.06048/) (2025)
- [John praised Mary because he? Implicit Causality Bias and Its Interaction with Explicit Cues in LMs](https://mlsanity.com/paper/2106.01060/) (2021)
- [Gaussian Process Surrogate Models for Neural Networks](https://mlsanity.com/paper/2208.06028/) (2022)
- [On a Benefit of Mask Language Modeling: Robustness to Simplicity Bias](https://mlsanity.com/paper/2110.05301/) (2021)
- [Is In-Context Learning a Type of Error-Driven Learning? Evidence from the Inverse Frequency Effect in Structural Priming](https://mlsanity.com/paper/2406.18501/) (2024)
- [Implicit Causality-biases in humans and LLMs as a tool for benchmarking LLM discourse capabilities](https://mlsanity.com/paper/2501.12980/) (2025)
- [semantic-features: A User-Friendly Tool for Studying Contextual Word Embeddings in Interpretable Semantic Spaces](https://mlsanity.com/paper/2506.06169/) (2025)
- [Meaning-infused grammar: Gradient Acceptability Shapes the Geometric Representations of Constructions in LLMs](https://mlsanity.com/paper/2507.22286/) (2025)
- [LLMs Learn Constructions That Humans Do Not Know](https://mlsanity.com/paper/2508.16837/) (2025)
- [Do Language Models Know What Not to Say? Causal Evidence for Statistical Preemption in LLMs](https://mlsanity.com/paper/2605.23039/) (2026)
- [A Scalable Pipeline for Estimating Verb Frame Frequencies Using Large Language Models](https://mlsanity.com/paper/2507.22187/) (2025)
- [Targeted Linguistic Analysis of Sign Language Models with Minimal Translation Pairs](https://mlsanity.com/paper/2604.27232/) (2026)
- [Causal Interventions on Continuous Variables: A Case Study on Verb Bias in Steering Vectors for In-Context Learning](https://mlsanity.com/paper/2605.29971/) (2026)

## Abstract (arXiv)

Languages typically provide more than one grammatical construction to express certain types of messages. A speaker's choice of construction is known to depend on multiple factors, including the choice of main verb -- a phenomenon known as verb bias. Here we introduce DAIS, a large benchmark dataset containing 50K human judgments for 5K distinct sentence pairs in the English dative alternation. This dataset includes 200 unique verbs and systematically varies the definiteness and length of arguments. We use this dataset, as well as an existing corpus of naturally occurring data, to evaluate how well recent neural language models capture human preferences. Results show that larger models perform better than smaller models, and transformer architectures (e.g. GPT-2) tend to out-perform recurrent architectures (e.g. LSTMs) even under comparable parameter and training settings. Additional analyses of internal feature representations suggest that transformers may better integrate specific lexical information with grammatical constructions.