# Evaluating Gender Bias in Natural Language Inference

arXiv: https://arxiv.org/abs/2105.05541 · HTML: https://mlsanity.com/paper/2105.05541/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 35 references that are in the map.
- First posted on arXiv 12 May 2021.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://mlsanity.com/paper/1902.01007/) (2019)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [Gender Bias in Coreference Resolution](https://mlsanity.com/paper/1804.09301/) (2018)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them](https://mlsanity.com/paper/1903.03862/) (2019)
- [Learning Gender-Neutral Word Embeddings](https://mlsanity.com/paper/1809.01496/) (2018)
- [Good-Enough Compositional Data Augmentation](https://mlsanity.com/paper/1904.09545/) (2019)
- [Reducing Gender Bias in Abusive Language Detection](https://mlsanity.com/paper/1808.07231/) (2018)
- [Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation](https://mlsanity.com/paper/1911.03842/) (2019)
- [Breaking NLI Systems with Sentences that Require Simple Lexical Inferences](https://mlsanity.com/paper/1805.02266/) (2018)
- [Dataset Augmentation in Feature Space](https://mlsanity.com/paper/1702.05538/) (2017)
- [Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual](https://mlsanity.com/paper/1908.10763/) (2019)
- [Understanding the Origins of Bias in Word Embeddings](https://mlsanity.com/paper/1810.03611/) (2018)
- [Ethical Challenges in Data-Driven Dialogue Systems](https://mlsanity.com/paper/1711.09050/) (2017)
- [Syntactic Data Augmentation Increases Robustness to Inference Heuristics](https://mlsanity.com/paper/2004.11999/) (2020)
- [Towards Robustifying NLI Models Against Lexical Dataset Biases](https://mlsanity.com/paper/2005.04732/) (2020)
- [Evaluating Models' Local Decision Boundaries via Contrast Sets](https://mlsanity.com/paper/2004.02709/) (2020)
- [Towards Understanding Gender Bias in Relation Extraction](https://mlsanity.com/paper/1911.03642/) (2019)

## Built on by (most cited first, 27 of 27)

- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [Theories of "Gender" in NLP Bias Research](https://mlsanity.com/paper/2205.02526/) (2022)
- [Perturbation Augmentation for Fairer NLP](https://mlsanity.com/paper/2205.12586/) (2022)
- [Fewer Errors, but More Stereotypes? The Effect of Model Size on Gender Bias](https://mlsanity.com/paper/2206.09860/) (2022)
- [Can LLMs Recognize Toxicity? A Structured Investigation Framework and Toxicity Metric](https://mlsanity.com/paper/2402.06900/) (2024)
- [Evaluating the Robustness of Discrete Prompts](https://mlsanity.com/paper/2302.05619/) (2023)
- [Evaluating Gender Bias of Pre-trained Language Models in Natural Language Inference by Considering All Labels](https://mlsanity.com/paper/2309.09697/) (2023)
- [What Makes it Ok to Set a Fire? Iterative Self-distillation of Contexts and Rationales for Disambiguating Defeasible Social and Moral Situations](https://mlsanity.com/paper/2310.15431/) (2023)
- [How Gender Debiasing Affects Internal Model Representations, and Why It Matters](https://mlsanity.com/paper/2204.06827/) (2022)
- [JASMINE: Arabic GPT Models for Few-Shot Learning](https://mlsanity.com/paper/2212.10755/) (2022)
- [How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contexts](https://mlsanity.com/paper/2205.10762/) (2022)
- [On Measuring Social Biases in Prompt-Based Multi-Task Learning](https://mlsanity.com/paper/2205.11605/) (2022)
- [Target-Agnostic Gender-Aware Contrastive Learning for Mitigating Bias in Multilingual Machine Translation](https://mlsanity.com/paper/2305.14016/) (2023)
- [Prompt Valuation Based on Shapley Values](https://mlsanity.com/paper/2312.15395/) (2023)
- [Mitigating Bias for Question Answering Models by Tracking Bias Influence](https://mlsanity.com/paper/2310.08795/) (2023)
- [FAST: Improving Controllability for Text Generation with Feedback Aware Self-Training](https://mlsanity.com/paper/2210.03167/) (2022)
- [Data Representativeness in Accessibility Datasets: A Meta-Analysis](https://mlsanity.com/paper/2207.08037/) (2022)
- [LawngNLI: A Long-Premise Benchmark for In-Domain Generalization from Short to Long Contexts and for Implication-Based Retrieval](https://mlsanity.com/paper/2212.03222/) (2022)
- [Tailoring with Targeted Precision: Edit-Based Agents for Open-Domain Procedure Customization](https://mlsanity.com/paper/2311.09510/) (2023)
- [Social Biases in Automatic Evaluation Metrics for NLG](https://mlsanity.com/paper/2210.08859/) (2022)
- [Disentangling Societal Inequality from Model Biases: Gender Inequality in Divorce Court Proceedings](https://mlsanity.com/paper/2307.10200/) (2023)
- [Learning to Predict Concept Ordering for Common Sense Generation](https://mlsanity.com/paper/2309.06363/) (2023)
- [Memory-assisted prompt editing to improve GPT-3 after deployment](https://mlsanity.com/paper/2201.06009/) (2022)
- [Subspace Representations for Soft Set Operations and Sentence Similarities](https://mlsanity.com/paper/2210.13034/) (2022)
- [Measuring Gender Bias in Job Title Matching for Grammatical Gender Languages](https://mlsanity.com/paper/2509.13803/) (2025)
- [Logically at Factify 2022: Multimodal Fact Verification](https://mlsanity.com/paper/2112.09253/) (2021)
- [AMR4NLI: Interpretable and robust NLI measures from semantic graphs](https://mlsanity.com/paper/2306.00936/) (2023)

## Abstract (arXiv)

Gender-bias stereotypes have recently raised significant ethical concerns in natural language processing. However, progress in detection and evaluation of gender bias in natural language understanding through inference is limited and requires further investigation. In this work, we propose an evaluation methodology to measure these biases by constructing a challenge task that involves pairing gender-neutral premises against a gender-specific hypothesis. We use our challenge task to investigate state-of-the-art NLI models on the presence of gender stereotypes using occupations. Our findings suggest that three models (BERT, RoBERTa, BART) trained on MNLI and SNLI datasets are significantly prone to gender-induced prediction errors. We also find that debiasing techniques such as augmenting the training dataset to ensure a gender-balanced dataset can help reduce such bias in certain cases.