# Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Models

arXiv: https://arxiv.org/abs/2205.11432 · HTML: https://mlsanity.com/paper/2205.11432/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 36 of its 51 references that are in the map.
- First posted on arXiv 23 May 2022.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 30 of 36)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://mlsanity.com/paper/1902.01007/) (2019)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Annotation Artifacts in Natural Language Inference Data](https://mlsanity.com/paper/1803.02324/) (2018)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [Hypothesis Only Baselines in Natural Language Inference](https://mlsanity.com/paper/1805.01042/) (2018)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Transformers as Soft Reasoners over Language](https://mlsanity.com/paper/2002.05867/) (2020)
- [Stress Test Evaluation for Natural Language Inference](https://mlsanity.com/paper/1806.00692/) (2018)
- [CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text](https://mlsanity.com/paper/1908.06177/) (2019)
- [End-to-End Differentiable Proving](https://mlsanity.com/paper/1705.11040/) (2017)
- [End-to-End Bias Mitigation by Modelling Biases in Corpora](https://mlsanity.com/paper/1909.06321/) (2019)
- [Towards Debiasing NLU Models from Unknown Biases](https://mlsanity.com/paper/2009.12303/) (2020)
- [NILE : Natural Language Inference with Faithful Natural Language Explanations](https://mlsanity.com/paper/2005.12116/) (2020)
- [Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance](https://mlsanity.com/paper/2005.00315/) (2020)
- [Learning Reasoning Strategies in End-to-End Differentiable Proving](https://mlsanity.com/paper/2007.06477/) (2020)
- [Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference](https://mlsanity.com/paper/1907.04380/) (2019)
- [NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language](https://mlsanity.com/paper/1906.06187/) (2019)
- [Supervising Model Attention with Human Explanations for Robust Natural Language Inference](https://mlsanity.com/paper/2104.08142/) (2021)
- [NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning](https://mlsanity.com/paper/2105.14167/) (2021)
- [What to Learn, and How: Toward Effective Learning from Rationales](https://mlsanity.com/paper/2112.00071/) (2021)
- [LIREx: Augmenting Language Inference with Relevant Explanation](https://mlsanity.com/paper/2012.09157/) (2020)
- [MonaLog: a Lightweight System for Natural Language Inference Based on Monotonicity](https://mlsanity.com/paper/1910.08772/) (2019)
- [LangPro: Natural Language Theorem Prover](https://mlsanity.com/paper/1708.09417/) (2017)
- [Zero-shot Sequence Labeling: Transferring Knowledge from Sentences to Tokens](https://mlsanity.com/paper/1805.02214/) (2018)
- [Exploring End-to-End Differentiable Natural Logic Modeling](https://mlsanity.com/paper/2011.04044/) (2020)
- [Weakly Supervised Explainable Phrasal Reasoning with Neural Fuzzy Logic](https://mlsanity.com/paper/2109.08927/) (2021)

## Built on by (most cited first, 18 of 18)

- [A Closer Look at Claim Decomposition](https://mlsanity.com/paper/2403.11903/) (2024)
- [On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMs](https://mlsanity.com/paper/2407.19200/) (2024)
- [Atomic Inference for NLI with Generated Facts as Atoms](https://mlsanity.com/paper/2305.13214/) (2023)
- [InfoLossQA: Characterizing and Recovering Information Loss in Text Simplification](https://mlsanity.com/paper/2401.16475/) (2024)
- [Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metrics](https://mlsanity.com/paper/2505.18658/) (2025)
- [NLI under the Microscope: What Atomic Hypothesis Decomposition Reveals](https://mlsanity.com/paper/2502.08080/) (2025)
- [Zero-Shot Fact-Checking with Semantic Triples and Knowledge Graphs](https://mlsanity.com/paper/2312.11785/) (2023)
- [Distilling Robustness into Natural Language Inference Models with Domain-Targeted Augmentation](https://mlsanity.com/paper/2305.13067/) (2023)
- [Zero-Shot Fact Verification via Natural Logic and Large Language Models](https://mlsanity.com/paper/2410.03341/) (2024)
- [Improving the OOD Performance of Closed-Source LLMs on NLI Through Strategic Data Selection](https://mlsanity.com/paper/2505.20209/) (2025)
- [X-PARADE: Cross-Lingual Textual Entailment and Information Divergence across Paragraphs](https://mlsanity.com/paper/2309.08873/) (2023)
- [Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More](https://mlsanity.com/paper/2503.10542/) (2025)
- [Extractive Fact Decomposition for Interpretable Natural Language Inference in one Forward Pass](https://mlsanity.com/paper/2509.18901/) (2025)
- [Enhancing adversarial robustness in Natural Language Inference using explanations](https://mlsanity.com/paper/2409.07423/) (2024)
- [Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainability](https://mlsanity.com/paper/2504.16056/) (2025)
- [Not Eliminate but Aggregate: Post-Hoc Control over Mixture-of-Experts to Address Shortcut Shifts in Natural Language Understanding](https://mlsanity.com/paper/2406.12060/) (2024)
- [Augmenting Legal Decision Support Systems with LLM-based NLI for Analyzing Social Media Evidence](https://mlsanity.com/paper/2410.15990/) (2024)
- [Learning Proposition-Level Links from Sentence-Level Supervision for Neuro-Symbolic NLI](https://mlsanity.com/paper/2610.15158/) (2026)

## Abstract (arXiv)

Current Natural Language Inference (NLI) models achieve impressive results, sometimes outperforming humans when evaluating on in-distribution test sets. However, as these models are known to learn from annotation artefacts and dataset biases, it is unclear to what extent the models are learning the task of NLI instead of learning from shallow heuristics in their training data. We address this issue by introducing a logical reasoning framework for NLI, creating highly transparent model decisions that are based on logical rules. Unlike prior work, we show that improved interpretability can be achieved without decreasing the predictive accuracy. We almost fully retain performance on SNLI, while also identifying the exact hypothesis spans that are responsible for each model prediction. Using the e-SNLI human explanations, we verify that our model makes sensible decisions at a span level, despite not using any span labels during training. We can further improve model performance and span-level decisions by using the e-SNLI explanations during training. Finally, our model is more robust in a reduced data setting. When training with only 1,000 examples, out-of-distribution performance improves on the MNLI matched and mismatched validation sets by 13% and 16% relative to the baseline. Training with fewer observations yields further improvements, both in-distribution and out-of-distribution.