mlsanityData to 7 Oct 2026

Light or Full Verb? A Minimal-Pair Dataset for Probing Phraseological Competence in Language Models

arXiv 2606.05087 ↗First posted 3 Jun 2026cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Frequent verbs such as 'have' and 'make' can function either as collocates in light-verb constructions or as full lexical predicates, as in 'make a decision' vs. 'make a cake'. Whether language models represent this distinction, and whether such representations vary across languages, remains unclear. We introduce a large-scale controlled dataset in English, Spanish, and French, comprising minimally varying sentence series in which the same context contains the same verb in light-verb and full-verb uses. Two probing experiments show that language models differentiate between these uses even in minimal contexts and exhibit separable patterns across object types. We release the dataset, generation code, and materials as a reusable resource. The framework supports extensions to broader contexts, additional verbs, and other languages.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 9 of its 41 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

9 of 41 references.

References · 9

  1. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  2. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 553
  3. 2018Targeted Syntactic Evaluation of Language Modelscited by 239
  4. 2020A Systematic Assessment of Syntactic Generalization in Neural Language Modelscited by 163
  5. 2025MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairscited by 65
  6. 2021CLiMP: A Benchmark for Chinese Language Model Evaluationcited by 44
  7. 2021A Targeted Assessment of Incremental Processing in Neural LanguageModels and Humanscited by 32
  8. 2021Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction from Language Modelscited by 15
  9. 2026Tracing the complexity profiles of different linguistic phenomena through the intrinsic dimension of LLM representationscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.