# Are LLMs Models of Distributional Semantics? A Case Study on Quantifiers

arXiv: https://arxiv.org/abs/2410.13984 · HTML: https://mlsanity.com/paper/2410.13984/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets](https://mlsanity.com/paper/2201.02177/) (2022)
- [Do Prompt-Based Models Really Understand the Meaning of their Prompts?](https://mlsanity.com/paper/2109.01247/) (2021)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [Binding Language Models in Symbolic Languages](https://mlsanity.com/paper/2210.02875/) (2022)
- [Learning Compositional Rules via Neural Program Synthesis](https://mlsanity.com/paper/2003.05562/) (2020)
- [Language models are not naysayers: An analysis of language models on negation benchmarks](https://mlsanity.com/paper/2306.08189/) (2023)
- [Distributional Semantics and Linguistic Theory](https://mlsanity.com/paper/1905.01896/) (2019)
- [Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge](https://mlsanity.com/paper/2305.05976/) (2023)
- [Combining Language and Vision with a Multimodal Skip-gram Model](https://mlsanity.com/paper/1501.02598/) (2015)
- [Improving negation detection with negation-focused pre-training](https://mlsanity.com/paper/2205.04012/) (2022)
- [Representing Meaning with a Combination of Logical and Distributional Models](https://mlsanity.com/paper/1505.06816/) (2015)
- [Probing Quantifier Comprehension in Large Language Models: Another Example of Inverse Scaling](https://mlsanity.com/paper/2306.07384/) (2023)
- [Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifiers](https://mlsanity.com/paper/2212.08700/) (2022)

## Built on by (most cited first, 3 of 3)

- [VAQUUM: Are Vague Quantifiers Grounded in Visual Data?](https://mlsanity.com/paper/2502.11874/) (2025)
- [Do Large Language Models Advocate for Inferentialism?](https://mlsanity.com/paper/2412.14501/) (2024)
- [Quantification and object perception in Multimodal Large Language Models and human linguistic cognition](https://mlsanity.com/paper/2511.08126/) (2025)

## Abstract (arXiv)

Distributional semantics is the linguistic theory that a word's meaning can be derived from its distribution in natural language (i.e., its use). Language models are commonly viewed as an implementation of distributional semantics, as they are optimized to capture the statistical features of natural language. It is often argued that distributional semantics models should excel at capturing graded/vague meaning based on linguistic conventions, but struggle with truth-conditional reasoning and symbolic processing. We evaluate this claim with a case study on vague (e.g. "many") and exact (e.g. "more than half") quantifiers. Contrary to expectations, we find that, across a broad range of models of various types, LLMs align more closely with human judgements on exact quantifiers versus vague ones. These findings call for a re-evaluation of the assumptions underpinning what distributional semantics models are, as well as what they can capture.