# ConjNLI: Natural Language Inference Over Conjunctive Sentences

arXiv: https://arxiv.org/abs/2010.10418 · HTML: https://mlsanity.com/paper/2010.10418/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2020.

## Built on by (most cited first, 27 of 27)

- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4](https://mlsanity.com/paper/2304.03439/) (2023)
- [Towards Faithful Model Explanation in NLP: A Survey](https://mlsanity.com/paper/2209.11326/) (2022)
- [Robustness Gym: Unifying the NLP Evaluation Landscape](https://mlsanity.com/paper/2101.04840/) (2021)
- [Natural Language Reasoning, A Survey](https://mlsanity.com/paper/2303.14725/) (2023)
- [IndoNLI: A Natural Language Inference Dataset for Indonesian](https://mlsanity.com/paper/2110.14566/) (2021)
- [When LLMs Meet Cunning Texts: A Fallacy Understanding Benchmark for Large Language Models](https://mlsanity.com/paper/2402.11100/) (2024)
- [Benchmarking Large Language Models for Persian: A Preliminary Study Focusing on ChatGPT](https://mlsanity.com/paper/2404.02403/) (2024)
- [A Neural-Symbolic Approach to Natural Language Understanding](https://mlsanity.com/paper/2203.10557/) (2022)
- [longhorns at DADC 2022: How many linguists does it take to fool a Question Answering model? A systematic approach to adversarial attacks](https://mlsanity.com/paper/2206.14729/) (2022)
- [SyGNS: A Systematic Generalization Testbed Based on Natural Language Semantics](https://mlsanity.com/paper/2106.01077/) (2021)
- [Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability](https://mlsanity.com/paper/2311.04449/) (2023)
- [Coordinate Constructions in English Enhanced Universal Dependencies: Analysis and Computational Modeling](https://mlsanity.com/paper/2103.08955/) (2021)
- [Investigating Transfer Learning in Multilingual Pre-trained Language Models through Chinese Natural Language Inference](https://mlsanity.com/paper/2106.03983/) (2021)
- [Efficient Beam Tree Recursion](https://mlsanity.com/paper/2307.10779/) (2023)
- [Benchmark Transparency: Measuring the Impact of Data on Evaluation](https://mlsanity.com/paper/2404.00748/) (2024)
- [How Hard is this Test Set? NLI Characterization by Exploiting Training Dynamics](https://mlsanity.com/paper/2410.03429/) (2024)
- [Testing Pre-trained Language Models' Understanding of Distributivity via Causal Mediation Analysis](https://mlsanity.com/paper/2209.04761/) (2022)
- [InferES : A Natural Language Inference Corpus for Spanish Featuring Negation-Based Contrastive and Adversarial Examples](https://mlsanity.com/paper/2210.03068/) (2022)
- [Inconsistent dialogue responses and how to recover from them](https://mlsanity.com/paper/2401.10353/) (2024)
- [ExpeTrans: LLMs Are Experiential Transfer Learners](https://mlsanity.com/paper/2505.23191/) (2025)
- [CANDLE: Decomposing Conditional and Conjunctive Queries for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2107.03884/) (2021)
- [On Degrees of Freedom in Defining and Testing Natural Language Understanding](https://mlsanity.com/paper/2305.15130/) (2023)
- [Formal Proofs as Structured Explanations: Proposing Several Tasks on Explainable Natural Language Inference](https://mlsanity.com/paper/2311.08637/) (2023)
- [CoRec: An Easy Approach for Coordination Recognition](https://mlsanity.com/paper/2311.18712/) (2023)
- [Self-Rationalization in the Wild: A Large Scale Out-of-Distribution Evaluation on NLI-related tasks](https://mlsanity.com/paper/2502.04797/) (2025)
- [KEHRL: Learning Knowledge-Enhanced Language Representations with Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2406.16374/) (2024)

## Abstract (arXiv)

Reasoning about conjuncts in conjunctive sentences is important for a deeper understanding of conjunctions in English and also how their usages and semantics differ from conjunctive and disjunctive boolean logic. Existing NLI stress tests do not consider non-boolean usages of conjunctions and use templates for testing such model knowledge. Hence, we introduce ConjNLI, a challenge stress-test for natural language inference over conjunctive sentences, where the premise differs from the hypothesis by conjuncts removed, added, or replaced. These sentences contain single and multiple instances of coordinating conjunctions ("and", "or", "but", "nor") with quantifiers, negations, and requiring diverse boolean and non-boolean inferences over conjuncts. We find that large-scale pre-trained language models like RoBERTa do not understand conjunctive semantics well and resort to shallow heuristics to make inferences over such sentences. As some initial solutions, we first present an iterative adversarial fine-tuning method that uses synthetically created training data based on boolean and non-boolean heuristics. We also propose a direct model advancement by making RoBERTa aware of predicate semantic roles. While we observe some performance gains, ConjNLI is still challenging for current methods, thus encouraging interesting future work for better understanding of conjunctions. Our data and code are publicly available at: https://github.com/swarnaHub/ConjNLI