# Let's CONFER: A Dataset for Evaluating Natural Language Inference Models on CONditional InFERence and Presupposition

arXiv: https://arxiv.org/abs/2506.06133 · HTML: https://mlsanity.com/paper/2506.06133/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 6 Jun 2025.

## Builds on (most cited first, 13 of 13)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference](https://mlsanity.com/paper/1808.05326/) (2018)
- [Abductive Commonsense Reasoning](https://mlsanity.com/paper/1908.05739/) (2019)
- [Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition](https://mlsanity.com/paper/2004.03066/) (2020)
- [Investigating BERT's Knowledge of Language: Five Analysis Methods with NPIs](https://mlsanity.com/paper/1909.02597/) (2019)
- [NOPE: A Corpus of Naturally-Occurring Presuppositions in English](https://mlsanity.com/paper/2109.06987/) (2021)
- [Let's do it "again": A First Computational Approach to Detecting Adverbial Presupposition Triggers](https://mlsanity.com/paper/1806.04262/) (2018)

## Built on by (most cited first, 3 of 3)

- [Do Language Models Know Theo Has a Wife? Investigating the Proviso Problem](https://mlsanity.com/paper/2603.08358/) (2026)
- [How Hypocritical Is Your LLM judge? Listener-Speaker Asymmetries in the Pragmatic Competence of Large Language Models](https://mlsanity.com/paper/2604.15873/) (2026)
- [Presupposition and Reasoning in Conditionals: A Theory-Based Study of Humans and LLMs](https://mlsanity.com/paper/2605.18352/) (2026)

## Abstract (arXiv)

Natural Language Inference (NLI) is the task of determining whether a sentence pair represents entailment, contradiction, or a neutral relationship. While NLI models perform well on many inference tasks, their ability to handle fine-grained pragmatic inferences, particularly presupposition in conditionals, remains underexplored. In this study, we introduce CONFER, a novel dataset designed to evaluate how NLI models process inference in conditional sentences. We assess the performance of four NLI models, including two pre-trained models, to examine their generalization to conditional reasoning. Additionally, we evaluate Large Language Models (LLMs), including GPT-4o, LLaMA, Gemma, and DeepSeek-R1, in zero-shot and few-shot prompting settings to analyze their ability to infer presuppositions with and without prior context. Our findings indicate that NLI models struggle with presuppositional reasoning in conditionals, and fine-tuning on existing NLI datasets does not necessarily improve their performance.