# Natural Language Inference in Context -- Investigating Contextual Reasoning over Long Texts

arXiv: https://arxiv.org/abs/2011.04864 · HTML: https://mlsanity.com/paper/2011.04864/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2020.

## Built on by (most cited first, 23 of 23)

- [Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4](https://mlsanity.com/paper/2304.03439/) (2023)
- [Natural Language Reasoning, A Survey](https://mlsanity.com/paper/2303.14725/) (2023)
- [AR-LSAT: Investigating Analytical Reasoning of Text](https://mlsanity.com/paper/2104.06598/) (2021)
- [Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text](https://mlsanity.com/paper/2105.03659/) (2021)
- [Towards Optimizing the Costs of LLM Usage](https://mlsanity.com/paper/2402.01742/) (2024)
- [Flexible Generation of Natural Language Deductions](https://mlsanity.com/paper/2104.08825/) (2021)
- [When LLMs Meet Cunning Texts: A Fallacy Understanding Benchmark for Large Language Models](https://mlsanity.com/paper/2402.11100/) (2024)
- [SemEval-2024 Task 2: Safe Biomedical Natural Language Inference for Clinical Trials](https://mlsanity.com/paper/2404.04963/) (2024)
- [Atomic Inference for NLI with Generated Facts as Atoms](https://mlsanity.com/paper/2305.13214/) (2023)
- [GLoRE: Evaluating Logical Reasoning of Large Language Models](https://mlsanity.com/paper/2310.09107/) (2023)
- [reStructured Pre-training](https://mlsanity.com/paper/2206.11147/) (2022)
- [Distilling Robustness into Natural Language Inference Models with Domain-Targeted Augmentation](https://mlsanity.com/paper/2305.13067/) (2023)
- [LawngNLI: A Long-Premise Benchmark for In-Domain Generalization from Short to Long Contexts and for Implication-Based Retrieval](https://mlsanity.com/paper/2212.03222/) (2022)
- [A Study on Training and Developing Large Language Models for Behavior Tree Generation](https://mlsanity.com/paper/2401.08089/) (2024)
- [LoNLI: An Extensible Framework for Testing Diverse Logical Reasoning Capabilities for NLI](https://mlsanity.com/paper/2112.02333/) (2021)
- [Curriculum: A Broad-Coverage Benchmark for Linguistic Phenomena in Natural Language Understanding](https://mlsanity.com/paper/2204.06283/) (2022)
- [Logic Agent: Enhancing Validity with Logic Rule Invocation](https://mlsanity.com/paper/2404.18130/) (2024)
- [Extractive Fact Decomposition for Interpretable Natural Language Inference in one Forward Pass](https://mlsanity.com/paper/2509.18901/) (2025)
- [Do Transformers know symbolic rules, and would we know if they did?](https://mlsanity.com/paper/2203.00162/) (2022)
- [SEval-Ex: A Statement-Level Framework for Explainable Summarization Evaluation](https://mlsanity.com/paper/2505.02235/) (2025)
- [DeKeyNLU: Enhancing Natural Language to SQL Generation through Task Decomposition and Keyword Extraction](https://mlsanity.com/paper/2509.14507/) (2025)
- [Towards Lighter and Robust Evaluation for Retrieval Augmented Generation](https://mlsanity.com/paper/2503.16161/) (2025)
- [SpecRead: A Benchmark for Measuring Whether Language Models Understand Hardware Specifications](https://mlsanity.com/paper/2609.33699/) (2026)

## Abstract (arXiv)

Natural language inference (NLI) is a fundamental NLP task, investigating the entailment relationship between two texts. Popular NLI datasets present the task at sentence-level. While adequate for testing semantic representations, they fall short for testing contextual reasoning over long texts, which is a natural part of the human inference process. We introduce ConTRoL, a new dataset for ConTextual Reasoning over Long texts. Consisting of 8,325 expert-designed "context-hypothesis" pairs with gold labels, ConTRoL is a passage-level NLI dataset with a focus on complex contextual reasoning types such as logical reasoning. It is derived from competitive selection and recruitment test (verbal reasoning test) for police recruitment, with expert level quality. Compared with previous NLI benchmarks, the materials in ConTRoL are much more challenging, involving a range of reasoning types. Empirical results show that state-of-the-art language models perform by far worse than educated humans. Our dataset can also serve as a testing-set for downstream tasks like Checking Factual Correctness of Summaries.