# Measuring Gender Bias in Job Title Matching for Grammatical Gender Languages

arXiv: https://arxiv.org/abs/2509.13803 · HTML: https://mlsanity.com/paper/2509.13803/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 27 references that are in the map.
- First posted on arXiv 17 Sep 2025.

## Builds on (most cited first, 23 of 23)

- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Gender Bias in Coreference Resolution](https://mlsanity.com/paper/1804.09301/) (2018)
- [On Measuring Social Biases in Sentence Encoders](https://mlsanity.com/paper/1903.10561/) (2019)
- [Multilingual E5 Text Embeddings: A Technical Report](https://mlsanity.com/paper/2402.05672/) (2024)
- [Evaluating Gender Bias in Machine Translation](https://mlsanity.com/paper/1906.00591/) (2019)
- [Multilingual Universal Sentence Encoder for Semantic Retrieval](https://mlsanity.com/paper/1907.04307/) (2019)
- [Assessing Gender Bias in Machine Translation -- A Case Study with Google Translate](https://mlsanity.com/paper/1809.02208/) (2018)
- [Gender Bias in Machine Translation](https://mlsanity.com/paper/2104.06001/) (2021)
- [Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias](https://mlsanity.com/paper/2010.14534/) (2020)
- [Fairness Testing: A Comprehensive Survey and Analysis of Trends](https://mlsanity.com/paper/2207.10223/) (2022)
- [On Measuring Gender Bias in Translation of Gender-neutral Pronouns](https://mlsanity.com/paper/1905.11684/) (2019)
- [Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer](https://mlsanity.com/paper/2005.00699/) (2020)
- [Gender Bias in Large Language Models across Multiple Languages](https://mlsanity.com/paper/2403.00277/) (2024)
- [JobBERT: Understanding Job Titles through Skills](https://mlsanity.com/paper/2109.09605/) (2021)
- [Evaluating Gender Bias in Natural Language Inference](https://mlsanity.com/paper/2105.05541/) (2021)
- [Gender Bias in Word Embeddings: A Comprehensive Analysis of Frequency, Syntax, and Semantics](https://mlsanity.com/paper/2206.03390/) (2022)
- [Good Secretaries, Bad Truck Drivers? Occupational Gender Stereotypes in Sentiment Analysis](https://mlsanity.com/paper/1906.10256/) (2019)
- [Embedding-based Recommender System for Job to Candidate Matching on Scale](https://mlsanity.com/paper/2107.00221/) (2021)
- [Learning Job Titles Similarity from Noisy Skill Labels](https://mlsanity.com/paper/2207.00494/) (2022)
- [Good, but not always Fair: An Evaluation of Gender Bias for three commercial Machine Translation Systems](https://mlsanity.com/paper/2306.05882/) (2023)

## Built on by (most cited first, 2 of 2)

- [Overview of the TalentCLEF 2026: Skill and Job Title Intelligence for Human Capital Management](https://mlsanity.com/paper/2606.31692/) (2026)
- [TalentCLEF at CLEF2026: Skill and Job Title Intelligence for Human Capital Management](https://mlsanity.com/paper/2607.20009/) (2026)

## Abstract (arXiv)

This work sets the ground for studying how explicit grammatical gender assignment in job titles can affect the results of automatic job ranking systems. We propose the usage of metrics for ranking comparison controlling for gender to evaluate gender bias in job title ranking systems, in particular RBO (Rank-Biased Overlap). We generate and share test sets for a job title matching task in four grammatical gender languages, including occupations in masculine and feminine form and annotated by gender and matching relevance. We use the new test sets and the proposed methodology to evaluate the gender bias of several out-of-the-box multilingual models to set as baselines, showing that all of them exhibit varying degrees of gender bias.