# Type B Reflexivization as an Unambiguous Testbed for Multilingual Multi-Task Gender Bias

arXiv: https://arxiv.org/abs/2009.11982 · HTML: https://mlsanity.com/paper/2009.11982/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2020.

## Built on by (most cited first, 15 of 15)

- [Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation](https://mlsanity.com/paper/2109.03858/) (2021)
- [Multi-Task Learning in Natural Language Processing: An Overview](https://mlsanity.com/paper/2109.09138/) (2021)
- [Assessing Multilingual Fairness in Pre-trained Multimodal Representations](https://mlsanity.com/paper/2106.06683/) (2021)
- [Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold](https://mlsanity.com/paper/2206.09755/) (2022)
- [Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models](https://mlsanity.com/paper/2109.03300/) (2021)
- [Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation](https://mlsanity.com/paper/2203.09866/) (2022)
- [Gender Bias in BERT -- Measuring and Analysing Biases through Sentiment Rating in a Realistic Downstream Classification Task](https://mlsanity.com/paper/2306.15298/) (2023)
- [Investigating Failures of Automatic Translation in the Case of Unambiguous Gender](https://mlsanity.com/paper/2104.07838/) (2021)
- [What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Study](https://mlsanity.com/paper/2410.00545/) (2024)
- [On the Interaction of Belief Bias and Explanations](https://mlsanity.com/paper/2106.15355/) (2021)
- [Multilingual large language models leak human stereotypes across language boundaries](https://mlsanity.com/paper/2312.07141/) (2023)
- [Are Pretrained Multilingual Models Equally Fair Across Languages?](https://mlsanity.com/paper/2210.05457/) (2022)
- [Gender, names and other mysteries: Towards the ambiguous for gender-inclusive translation](https://mlsanity.com/paper/2306.04573/) (2023)
- [Sometimes We Want Translationese](https://mlsanity.com/paper/2104.07623/) (2021)
- [Are Female Carpenters like Blue Bananas? A Corpus Investigation of Occupation Gender Typicality](https://mlsanity.com/paper/2408.02948/) (2024)

## Abstract (arXiv)

The one-sided focus on English in previous studies of gender bias in NLP misses out on opportunities in other languages: English challenge datasets such as GAP and WinoGender highlight model preferences that are "hallucinatory", e.g., disambiguating gender-ambiguous occurrences of 'doctor' as male doctors. We show that for languages with type B reflexivization, e.g., Swedish and Russian, we can construct multi-task challenge datasets for detecting gender bias that lead to unambiguously wrong model predictions: In these languages, the direct translation of 'the doctor removed his mask' is not ambiguous between a coreferential reading and a disjoint reading. Instead, the coreferential reading requires a non-gendered pronoun, and the gendered, possessive pronouns are anti-reflexive. We present a multilingual, multi-task challenge dataset, which spans four languages and four NLP tasks and focuses only on this phenomenon. We find evidence for gender bias across all task-language combinations and correlate model bias with national labor market statistics.