Measuring Gender Bias in Job Title Matching for Grammatical Gender Languages
Not on the map. Why?
Abstract (arXiv)
This work sets the ground for studying how explicit grammatical gender assignment in job titles can affect the results of automatic job ranking systems. We propose the usage of metrics for ranking comparison controlling for gender to evaluate gender bias in job title ranking systems, in particular RBO (Rank-Biased Overlap). We generate and share test sets for a job title matching task in four grammatical gender languages, including occupations in masculine and feminine form and annotated by gender and matching relevance. We use the new test sets and the proposed methodology to evaluate the gender bias of several out-of-the-box multilingual models to set as baselines, showing that all of them exhibit varying degrees of gender bias.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 23 of its 27 references that are in the map. First posted on arXiv 17 Sep 2025.
Builds on
23 of 27 references.
References · 23
- 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
- 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
- 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
- 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 659
- 2018Gender Bias in Coreference Resolutioncited by 442
- 2019On Measuring Social Biases in Sentence Encoderscited by 398
- 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 372
- 2019Evaluating Gender Bias in Machine Translationcited by 249
- 2019Multilingual Universal Sentence Encoder for Semantic Retrievalcited by 193
- 2018Assessing Gender Bias in Machine Translation -- A Case Study with Google Translatecited by 121
- 2021Gender Bias in Machine Translationcited by 117
- 2020Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Biascited by 82
11 more
- 2022Fairness Testing: A Comprehensive Survey and Analysis of Trendscited by 62
- 2019On Measuring Gender Bias in Translation of Gender-neutral Pronounscited by 60
- 2020Gender Bias in Multilingual Embeddings and Cross-Lingual Transfercited by 55
- 2024Gender Bias in Large Language Models across Multiple Languagescited by 35
- 2021JobBERT: Understanding Job Titles through Skillscited by 29
- 2021Evaluating Gender Bias in Natural Language Inferencecited by 27
- 2022Gender Bias in Word Embeddings: A Comprehensive Analysis of Frequency, Syntax, and Semanticscited by 24
- 2019Good Secretaries, Bad Truck Drivers? Occupational Gender Stereotypes in Sentiment Analysiscited by 19
- 2021Embedding-based Recommender System for Job to Candidate Matching on Scalecited by 12
- 2022Learning Job Titles Similarity from Noisy Skill Labelscited by 12
- 2023Good, but not always Fair: An Evaluation of Gender Bias for three commercial Machine Translation Systemscited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.