# Man is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition

arXiv: https://arxiv.org/abs/1910.10872 · HTML: https://mlsanity.com/paper/1910.10872/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2019.

## Built on by (most cited first, 28 of 28)

- [Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies](https://mlsanity.com/paper/2108.12084/) (2021)
- [Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics](https://mlsanity.com/paper/2106.14574/) (2021)
- [On Measures of Biases and Harms in NLP](https://mlsanity.com/paper/2108.03362/) (2021)
- [Large Language Models and Knowledge Graphs: Opportunities and Challenges](https://mlsanity.com/paper/2308.06374/) (2023)
- [Theories of "Gender" in NLP Bias Research](https://mlsanity.com/paper/2205.02526/) (2022)
- [Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources](https://mlsanity.com/paper/2103.11320/) (2021)
- [NewsEdits: A News Article Revision Dataset and a Document-Level Reasoning Challenge](https://mlsanity.com/paper/2206.07106/) (2022)
- [Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies](https://mlsanity.com/paper/2312.11779/) (2023)
- [Examining Covert Gender Bias: A Case Study in Turkish and English Machine Translation Models](https://mlsanity.com/paper/2108.10379/) (2021)
- [The Arabic Parallel Gender Corpus 2.0: Extensions and Analyses](https://mlsanity.com/paper/2110.09216/) (2021)
- [Does Data Repair Lead to Fair Models? Curating Contextually Fair Data To Reduce Model Bias](https://mlsanity.com/paper/2110.10389/) (2021)
- [In the Name of Fairness: Assessing the Bias in Clinical Record De-identification](https://mlsanity.com/paper/2305.11348/) (2023)
- [Towards Multi-Objective Statistically Fair Federated Learning](https://mlsanity.com/paper/2201.09917/) (2022)
- [Examining the Causal Effect of First Names on Language Models: The Case of Social Commonsense Reasoning](https://mlsanity.com/paper/2306.01117/) (2023)
- [Mitigating Bias for Question Answering Models by Tracking Bias Influence](https://mlsanity.com/paper/2310.08795/) (2023)
- [User-Centric Gender Rewriting](https://mlsanity.com/paper/2205.02211/) (2022)
- [ER-Test: Evaluating Explanation Regularization Methods for Language Models](https://mlsanity.com/paper/2205.12542/) (2022)
- [Measuring Geographic Performance Disparities of Offensive Language Classifiers](https://mlsanity.com/paper/2209.07353/) (2022)
- [The Lifecycle of "Facts": A Survey of Social Bias in Knowledge Graphs](https://mlsanity.com/paper/2210.03353/) (2022)
- [A Comprehensive Study of Gender Bias in Chemical Named Entity Recognition Models](https://mlsanity.com/paper/2212.12799/) (2022)
- [Do Existing Testing Tools Really Uncover Gender Bias in Text-to-Image Models?](https://mlsanity.com/paper/2501.15775/) (2025)
- [Crowdsourcing on Sensitive Data with Privacy-Preserving Text Rewriting](https://mlsanity.com/paper/2303.03053/) (2023)
- [BEADs: Bias Evaluation Across Domains](https://mlsanity.com/paper/2406.04220/) (2024)
- [NewsEdits: A Dataset of Revision Histories for News Articles (Technical Report: Data Processing)](https://mlsanity.com/paper/2104.09647/) (2021)
- [National Origin Discrimination in Deep-learning-powered Automated Resume Screening](https://mlsanity.com/paper/2307.08624/) (2023)
- [Towards Fair and Explainable AI using a Human-Centered AI Approach](https://mlsanity.com/paper/2306.07427/) (2023)
- [Large Language Models for Geolocation Extraction in Humanitarian Crisis Response](https://mlsanity.com/paper/2602.08872/) (2026)
- [An Approach to Ensure Fairness in News Articles](https://mlsanity.com/paper/2207.03938/) (2022)

## Abstract (arXiv)

We study the bias in several state-of-the-art named entity recognition (NER) models---specifically, a difference in the ability to recognize male and female names as PERSON entity types. We evaluate NER models on a dataset containing 139 years of U.S. census baby names and find that relatively more female names, as opposed to male names, are not recognized as PERSON entities. We study the extent of this bias in several NER systems that are used prominently in industry and academia. In addition, we also report a bias in the datasets on which these models were trained. The result of this analysis yields a new benchmark for gender bias evaluation in named entity recognition systems. The data and code for the application of this benchmark will be publicly available for researchers to use.