# NOPE: A Corpus of Naturally-Occurring Presuppositions in English

arXiv: https://arxiv.org/abs/2109.06987 · HTML: https://mlsanity.com/paper/2109.06987/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 71 references that are in the map.
- First posted on arXiv 14 Sep 2021.

## Builds on (most cited first, 17 of 17)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://mlsanity.com/paper/1902.01007/) (2019)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [Advances in Pre-Training Distributed Word Representations](https://mlsanity.com/paper/1712.09405/) (2017)
- [Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition](https://mlsanity.com/paper/2004.03066/) (2020)
- [Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering](https://mlsanity.com/paper/2101.00391/) (2021)
- [Uncertain Natural Language Inference](https://mlsanity.com/paper/1909.03042/) (2019)
- [Harnessing the linguistic signal to predict scalar inferences](https://mlsanity.com/paper/1910.14254/) (2019)
- [Lexicosyntactic Inference in Neural Models](https://mlsanity.com/paper/1808.06232/) (2018)

## Built on by (most cited first, 23 of 23)

- [Understanding Transformer Memorization Recall Through Idioms](https://mlsanity.com/paper/2210.03588/) (2022)
- [PUB: A Pragmatics Understanding Benchmark for Assessing LLMs' Pragmatics Capabilities](https://mlsanity.com/paper/2401.07078/) (2024)
- [Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMs](https://mlsanity.com/paper/2403.05020/) (2024)
- [(QA)2: Question Answering with Questionable Assumptions](https://mlsanity.com/paper/2212.10003/) (2022)
- [Benchmarks for Automated Commonsense Reasoning: A Survey](https://mlsanity.com/paper/2302.04752/) (2023)
- [SEA-HELM: Southeast Asian Holistic Evaluation of Language Models](https://mlsanity.com/paper/2502.14301/) (2025)
- [BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models](https://mlsanity.com/paper/2309.06085/) (2023)
- [Pregnant Questions: The Importance of Pragmatic Awareness in Maternal Health Question Answering](https://mlsanity.com/paper/2311.09542/) (2023)
- [Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approaches](https://mlsanity.com/paper/2211.08371/) (2022)
- [Linguistic Generalizations are not Rules: Impacts on Evaluation of LMs](https://mlsanity.com/paper/2502.13195/) (2025)
- [Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning](https://mlsanity.com/paper/2506.08354/) (2025)
- [Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds](https://mlsanity.com/paper/2305.14785/) (2023)
- [Evaluating Paraphrastic Robustness in Textual Entailment Models](https://mlsanity.com/paper/2306.16722/) (2023)
- [Experimental Pragmatics with Machines: Testing LLM Predictions for the Inferences of Plain and Embedded Disjunctions](https://mlsanity.com/paper/2405.05776/) (2024)
- [Towards Understanding What Code Language Models Learned](https://mlsanity.com/paper/2306.11943/) (2023)
- [Let's CONFER: A Dataset for Evaluating Natural Language Inference Models on CONditional InFERence and Presupposition](https://mlsanity.com/paper/2506.06133/) (2025)
- [Do Language Models Know Theo Has a Wife? Investigating the Proviso Problem](https://mlsanity.com/paper/2603.08358/) (2026)
- [It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief](https://mlsanity.com/paper/2607.18232/) (2026)
- [Polish Natural Language Inference and Factivity -- an Expert-based Dataset and Benchmarks](https://mlsanity.com/paper/2201.03521/) (2022)
- [PROPRES: Investigating the Projectivity of Presupposition with Various Triggers and Environments](https://mlsanity.com/paper/2312.08755/) (2023)
- [LIEDER: Linguistically-Informed Evaluation for Discourse Entity Recognition](https://mlsanity.com/paper/2403.06301/) (2024)
- [DRInQ: Evaluating Conversational Implicature with Controlled Context Variation](https://mlsanity.com/paper/2605.24267/) (2026)
- [Pragmatic Attack Surface: Vulnerabilities of Implicit Context in Large Language Models](https://mlsanity.com/paper/2608.09551/) (2026)

## Abstract (arXiv)

Understanding language requires grasping not only the overtly stated content, but also making inferences about things that were left unsaid. These inferences include presuppositions, a phenomenon by which a listener learns about new information through reasoning about what a speaker takes as given. Presuppositions require complex understanding of the lexical and syntactic properties that trigger them as well as the broader conversational context. In this work, we introduce the Naturally-Occurring Presuppositions in English (NOPE) Corpus to investigate the context-sensitivity of 10 different types of presupposition triggers and to evaluate machine learning models' ability to predict human inferences. We find that most of the triggers we investigate exhibit moderate variability. We further find that transformer-based models draw correct inferences in simple cases involving presuppositions, but they fail to capture the minority of exceptional cases in which human judgments reveal complex interactions between context and triggers.