Neither True nor False: Do LLMs Represent Truth-Value Gaps in Plural Predication?
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) are increasingly being used to summarize and verify claims in critical sources that inform consequential decisions. Such claims are full of plural definites. If three of five patients improved after taking drug A, the claim the patients improved after drug A is not true, since two did not, yet its negation, the patients didn't improve after drug A, is not true either, since three did. In formal semantics, this truth-value gap is known as the Homogeneity Gap: both the plural definite and its negation are neither true nor false when the property holds for only a fraction of the plural referent. A model that cannot represent this gap turns mixed evidence into a clean verdict, an error that can be costly in high-stakes use. We introduce HomogeneityNLI, a diagnostic dataset of 1,000 minimal pair items crossing determiner, polarity, and group composition across ten logical conditions. Evaluating seven open-weight models, we uncover a pervasive Bivalence Bias: models assign plural definites their homogeneous truth conditions. In the zero-shot setting, no model recognizes the gap for both polarities in more than 15% of scenarios, and negated definites are judged False in 84% of cases. Explicit linguistic rules, chain-of-thought, and enumeration prompts produce unstable, model-specific effects that often entrench the bias. Our findings indicate that current open-weight LLMs acquire the truth conditions of plural predication but not its trivalent logic, and cannot yet be trusted to summarize or verify claims whose evidence is mixed.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 37 references that are in the map. First posted on arXiv 10 Oct 2026.
Builds on
14 of 37 references.
References · 14
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
- 2015A large annotated corpus for learning natural language inferencecited by 1,847
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
- 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
- 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 876
- 2024Yi: Open Foundation Models by 01.AIcited by 719
- 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
- 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
- 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 320
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025MEDEQUALQA: Evaluating Biases in LLMs with Counterfactual Reasoning
- 2026The strength of clinical evidence is recoverable from language model representations but not from their stated grades
- 2025Are the Hidden States Hiding Something? Testing the Limits of Factuality-Encoding Capabilities in LLMs
- 2026The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Shared Category Geometry in Small Language Models
- 2023LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond
- 2024A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners
- 2026CLEAR: Revealing How Noise and Ambiguity Degrade Reliability in LLMs for Medicine
- 2026Presupposition and Reasoning in Conditionals: A Theory-Based Study of Humans and LLMs