mlsanityData to 7 Oct 2026

Reverse-engineering NLI: A study of the meta-inferential properties of Natural Language Inference

arXiv 2601.05170 ↗First posted 8 Jan 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Natural Language Inference (NLI) has been an important task for evaluating language models for Natural Language Understanding, but the logical properties of the task are poorly understood and often mischaracterized. Understanding the notion of inference captured by NLI is key to interpreting model performance on the task. In this paper we formulate three possible readings of the NLI label set and perform a comprehensive analysis of the meta-inferential properties they entail. Focusing on the SNLI dataset, we exploit (1) NLI items with shared premises and (2) items generated by LLMs to evaluate models trained on SNLI for meta-inferential consistency and derive insights into which reading of the logical relations is encoded by the dataset.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 38 references that are in the map. First posted on arXiv 8 Jan 2026.

Builds on

18 of 38 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  5. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  6. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  7. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  8. 2018Hypothesis Only Baselines in Natural Language Inferencecited by 291
  9. 2018Lessons from Natural Language Inference in the Clinical Domaincited by 149
  10. 2019A Logic-Driven Framework for Consistency of Neural Modelscited by 64
  11. 2018Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledgecited by 45
  12. 2020Self-Explaining Structures Improve NLP Modelscited by 22
6 more
  1. 2018What If We Simply Swap the Two Text Fragments? A Straightforward yet Effective Way to Test the Robustness of Methods to Confounding Signals in Nature Language Inference Taskscited by 14
  2. 2022Logical Satisfiability of Counterfactuals for Faithful Explanations in NLIcited by 11
  3. 2024Conditional and Modal Reasoning in Large Language Modelscited by 9
  4. 2025NLI under the Microscope: What Atomic Hypothesis Decomposition Revealscited by 9
  5. 2024First Train to Generate, then Generate to Train: UnitedSynT5 for Few-Shot NLIcited by 7
  6. 2024Identification of Entailment and Contradiction Relations between Natural Language Sentences: A Neurosymbolic Approachcited by 2

Built on by

Not cited yet.