# Supporting Context Monotonicity Abstractions in Neural NLI Models

arXiv: https://arxiv.org/abs/2105.08008 · HTML: https://mlsanity.com/paper/2105.08008/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Probing Natural Language Inference Models through Semantic Fragments](https://mlsanity.com/paper/1909.07521/) (2019)
- [Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets](https://mlsanity.com/paper/1904.02668/) (2019)
- [Probing Linguistic Systematicity](https://mlsanity.com/paper/2005.04315/) (2020)
- [HELP: A Dataset for Identifying Shortcomings of Neural Models in Monotonicity Reasoning](https://mlsanity.com/paper/1904.12166/) (2019)
- [Do Neural Models Learn Systematicity of Monotonicity Inference in Natural Language?](https://mlsanity.com/paper/2004.14839/) (2020)
- [MonaLog: a Lightweight System for Natural Language Inference Based on Monotonicity](https://mlsanity.com/paper/1910.08772/) (2019)

## Built on by (most cited first, 5 of 5)

- [Systematicity, Compositionality and Transitivity of Deep NLP Models: a Metamorphic Testing Perspective](https://mlsanity.com/paper/2204.12316/) (2022)
- [Interventional Probing in High Dimensions: An NLI Case Study](https://mlsanity.com/paper/2304.10346/) (2023)
- [Decomposing Natural Logic Inferences in Neural NLI](https://mlsanity.com/paper/2112.08289/) (2021)
- [Estimating the Causal Effects of Natural Logic Features in Neural NLI Models](https://mlsanity.com/paper/2305.08572/) (2023)
- [Estimating the Causal Effects of Natural Logic Features in Transformer-Based NLI Models](https://mlsanity.com/paper/2404.02622/) (2024)

## Abstract (arXiv)

Natural language contexts display logical regularities with respect to substitutions of related concepts: these are captured in a functional order-theoretic property called monotonicity. For a certain class of NLI problems where the resulting entailment label depends only on the context monotonicity and the relation between the substituted concepts, we build on previous techniques that aim to improve the performance of NLI models for these problems, as consistent performance across both upward and downward monotone contexts still seems difficult to attain even for state-of-the-art models. To this end, we reframe the problem of context monotonicity classification to make it compatible with transformer-based pre-trained NLI models and add this task to the training pipeline. Furthermore, we introduce a sound and complete simplified monotonicity logic formalism which describes our treatment of contexts as abstract units. Using the notions in our formalism, we adapt targeted challenge sets to investigate whether an intermediate context monotonicity classification task can aid NLI models' performance on examples exhibiting monotonicity reasoning.