# Towards Understanding Gender Bias in Relation Extraction

arXiv: https://arxiv.org/abs/1911.03642 · HTML: https://mlsanity.com/paper/1911.03642/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2019.

## Built on by (most cited first, 23 of 23)

- [Large Language Model Alignment: A Survey](https://mlsanity.com/paper/2309.15025/) (2023)
- [Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics](https://mlsanity.com/paper/2106.14574/) (2021)
- [On Measures of Biases and Harms in NLP](https://mlsanity.com/paper/2108.03362/) (2021)
- [Theories of "Gender" in NLP Bias Research](https://mlsanity.com/paper/2205.02526/) (2022)
- [On Learning Fairness and Accuracy on Multiple Subgroups](https://mlsanity.com/paper/2210.10837/) (2022)
- [Sociodemographic Bias in Language Models: A Survey and Forward Path](https://mlsanity.com/paper/2306.08158/) (2023)
- [Evaluating Gender Bias in Natural Language Inference](https://mlsanity.com/paper/2105.05541/) (2021)
- [Twists, Humps, and Pebbles: Multilingual Speech Recognition Models Exhibit Gender Performance Gaps](https://mlsanity.com/paper/2402.17954/) (2024)
- [ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Models](https://mlsanity.com/paper/2310.09624/) (2023)
- [Watching the Watchers: Exposing Gender Disparities in Machine Translation Quality Estimation](https://mlsanity.com/paper/2410.10995/) (2024)
- [ChatGPT is on the Horizon: Could a Large Language Model be Suitable for Intelligent Traffic Safety Research and Applications?](https://mlsanity.com/paper/2303.05382/) (2023)
- [Mitigating Bias for Question Answering Models by Tracking Bias Influence](https://mlsanity.com/paper/2310.08795/) (2023)
- [Mitigating Social Bias in Large Language Models: A Multi-Objective Approach within a Multi-Agent Framework](https://mlsanity.com/paper/2412.15504/) (2024)
- [Trustworthy Social Bias Measurement](https://mlsanity.com/paper/2212.11672/) (2022)
- [Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairness](https://mlsanity.com/paper/2303.00673/) (2023)
- [Learning Stable Classifiers by Transferring Unstable Features](https://mlsanity.com/paper/2106.07847/) (2021)
- [Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives](https://mlsanity.com/paper/2506.09656/) (2025)
- [The Lifecycle of "Facts": A Survey of Social Bias in Knowledge Graphs](https://mlsanity.com/paper/2210.03353/) (2022)
- [A Comprehensive Study of Gender Bias in Chemical Named Entity Recognition Models](https://mlsanity.com/paper/2212.12799/) (2022)
- [Formulation Comparison for Timeline Construction using LLMs](https://mlsanity.com/paper/2403.00990/) (2024)
- [Human Bias in the Face of AI: Examining Human Judgment Against Text Labeled as AI Generated](https://mlsanity.com/paper/2410.03723/) (2024)
- [From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Text](https://mlsanity.com/paper/2508.09450/) (2025)
- [Challenges in Annotating Datasets to Quantify Bias in Under-represented Society](https://mlsanity.com/paper/2309.08624/) (2023)

## Abstract (arXiv)

Recent developments in Neural Relation Extraction (NRE) have made significant strides towards Automated Knowledge Base Construction (AKBC). While much attention has been dedicated towards improvements in accuracy, there have been no attempts in the literature to our knowledge to evaluate social biases in NRE systems. We create WikiGenderBias, a distantly supervised dataset with a human annotated test set. WikiGenderBias has sentences specifically curated to analyze gender bias in relation extraction systems. We use WikiGenderBias to evaluate systems for bias and find that NRE systems exhibit gender biased predictions and lay groundwork for future evaluation of bias in NRE. We also analyze how name anonymization, hard debiasing for word embeddings, and counterfactual data augmentation affect gender bias in predictions and performance.