# HypoNLI: Exploring the Artificial Patterns of Hypothesis-only Bias in Natural Language Inference

arXiv: https://arxiv.org/abs/2003.02756 · HTML: https://mlsanity.com/paper/2003.02756/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Mar 2020.

## Built on by (most cited first, 17 of 17)

- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Elements of World Knowledge (EWoK): A Cognition-Inspired Framework for Evaluating Basic World Knowledge in Language Models](https://mlsanity.com/paper/2405.09605/) (2024)
- [Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets](https://mlsanity.com/paper/2203.12942/) (2022)
- [DISCO: Distilling Counterfactuals with Large Language Models](https://mlsanity.com/paper/2212.10534/) (2022)
- [Sociodemographic Bias in Language Models: A Survey and Forward Path](https://mlsanity.com/paper/2306.08158/) (2023)
- [When Does Group Invariant Learning Survive Spurious Correlations?](https://mlsanity.com/paper/2206.14534/) (2022)
- [Uncertainty Calibration for Ensemble-Based Debiasing Methods](https://mlsanity.com/paper/2111.04104/) (2021)
- [AGRO: Adversarial Discovery of Error-prone groups for Robust Optimization](https://mlsanity.com/paper/2212.00921/) (2022)
- [Behind the Scenes: An Exploration of Trigger Biases Problem in Few-Shot Event Classification](https://mlsanity.com/paper/2108.12844/) (2021)
- [DialogVCS: Robust Natural Language Understanding in Dialogue System Upgrade](https://mlsanity.com/paper/2305.14751/) (2023)
- [SETI: Systematicity Evaluation of Textual Inference](https://mlsanity.com/paper/2305.15045/) (2023)
- [How Hard is this Test Set? NLI Characterization by Exploiting Training Dynamics](https://mlsanity.com/paper/2410.03429/) (2024)
- [SMoA: Sparse Mixture of Adapters to Mitigate Multiple Dataset Biases](https://mlsanity.com/paper/2302.14413/) (2023)
- [Using Natural Language Explanations to Improve Robustness of In-context Learning](https://mlsanity.com/paper/2311.07556/) (2023)
- [Exploring Explanations Improves the Robustness of In-Context Learning](https://mlsanity.com/paper/2506.02378/) (2025)
- [Out-of-Distribution Generalization in Text Classification: Past, Present, and Future](https://mlsanity.com/paper/2305.14104/) (2023)
- [Evaluating the Scalability and Adversarial Generalization of GRPO-Trained NLI Models](https://mlsanity.com/paper/2504.18376/) (2025)

## Abstract (arXiv)

Many recent studies have shown that for models trained on datasets for natural language inference (NLI), it is possible to make correct predictions by merely looking at the hypothesis while completely ignoring the premise. In this work, we manage to derive adversarial examples in terms of the hypothesis-only bias and explore eligible ways to mitigate such bias. Specifically, we extract various phrases from the hypotheses (artificial patterns) in the training sets, and show that they have been strong indicators to the specific labels. We then figure out `hard' and `easy' instances from the original test sets whose labels are opposite to or consistent with those indications. We also set up baselines including both pretrained models (BERT, RoBERTa, XLNet) and competitive non-pretrained models (InferSent, DAM, ESIM). Apart from the benchmark and baselines, we also investigate two debiasing approaches which exploit the artificial pattern modeling to mitigate such hypothesis-only bias: down-sampling and adversarial training. We believe those methods can be treated as competitive baselines in NLI debiasing tasks.