# Reverse-engineering NLI: A study of the meta-inferential properties of Natural Language Inference

arXiv: https://arxiv.org/abs/2601.05170 · HTML: https://mlsanity.com/paper/2601.05170/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 8 Jan 2026.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [XNLI: Evaluating Cross-lingual Sentence Representations](https://mlsanity.com/paper/1809.05053/) (2018)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [Hypothesis Only Baselines in Natural Language Inference](https://mlsanity.com/paper/1805.01042/) (2018)
- [Lessons from Natural Language Inference in the Clinical Domain](https://mlsanity.com/paper/1808.06752/) (2018)
- [A Logic-Driven Framework for Consistency of Neural Models](https://mlsanity.com/paper/1909.00126/) (2019)
- [Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge](https://mlsanity.com/paper/1808.08609/) (2018)
- [Self-Explaining Structures Improve NLP Models](https://mlsanity.com/paper/2012.01786/) (2020)
- [What If We Simply Swap the Two Text Fragments? A Straightforward yet Effective Way to Test the Robustness of Methods to Confounding Signals in Nature Language Inference Tasks](https://mlsanity.com/paper/1809.02719/) (2018)
- [Logical Satisfiability of Counterfactuals for Faithful Explanations in NLI](https://mlsanity.com/paper/2205.12469/) (2022)
- [NLI under the Microscope: What Atomic Hypothesis Decomposition Reveals](https://mlsanity.com/paper/2502.08080/) (2025)
- [Conditional and Modal Reasoning in Large Language Models](https://mlsanity.com/paper/2401.17169/) (2024)
- [First Train to Generate, then Generate to Train: UnitedSynT5 for Few-Shot NLI](https://mlsanity.com/paper/2412.09263/) (2024)
- [Identification of Entailment and Contradiction Relations for Natural Language Argumentation: A Neurosymbolic Approach](https://mlsanity.com/paper/2405.01259/) (2024)

## Abstract (arXiv)

Natural Language Inference (NLI) has been an important task for evaluating language models for Natural Language Understanding, but the logical properties of the task are poorly understood and often mischaracterized. Understanding the notion of inference captured by NLI is key to interpreting model performance on the task. In this paper we formulate three possible readings of the NLI label set and perform a comprehensive analysis of the meta-inferential properties they entail. Focusing on the SNLI dataset, we exploit (1) NLI items with shared premises and (2) items generated by LLMs to evaluate models trained on SNLI for meta-inferential consistency and derive insights into which reading of the logical relations is encoded by the dataset.