# GATE X-E : A Challenge Set for Gender-Fair Translations from Weakly-Gendered Languages

arXiv: https://arxiv.org/abs/2402.14277 · HTML: https://mlsanity.com/paper/2402.14277/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 22 Feb 2024.

## Builds on (most cited first, 18 of 18)

- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Evaluating Gender Bias in Machine Translation](https://mlsanity.com/paper/1906.00591/) (2019)
- [WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia](https://mlsanity.com/paper/1907.05791/) (2019)
- [Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters](https://mlsanity.com/paper/2212.10001/) (2022)
- [CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs](https://mlsanity.com/paper/1911.06154/) (2019)
- [Assessing Gender Bias in Machine Translation -- A Case Study with Google Translate](https://mlsanity.com/paper/1809.02208/) (2018)
- [Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem](https://mlsanity.com/paper/2004.04498/) (2020)
- [Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpus](https://mlsanity.com/paper/2006.05754/) (2020)
- [QED: A Framework and Dataset for Explanations in Question Answering](https://mlsanity.com/paper/2009.06354/) (2020)
- [NeuTral Rewriter: A Rule-Based and Neural Approach to Automatic Rewriting into Gender-Neutral Alternatives](https://mlsanity.com/paper/2109.06105/) (2021)
- [They, Them, Theirs: Rewriting with Gender-Neutral English](https://mlsanity.com/paper/2102.06788/) (2021)
- [MT-GenEval: A Counterfactual and Contextual Dataset for Evaluating Gender Accuracy in Machine Translation](https://mlsanity.com/paper/2211.01355/) (2022)
- [Gender Bias Amplification During Speed-Quality Optimization in Neural Machine Translation](https://mlsanity.com/paper/2106.00169/) (2021)
- [Multilingual Holistic Bias: Extending Descriptors and Patterns to Unveil Demographic Biases in Languages at Scale](https://mlsanity.com/paper/2305.13198/) (2023)
- [Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation](https://mlsanity.com/paper/2203.09866/) (2022)
- [Gender Neutralization for an Inclusive Machine Translation: from Theoretical Foundations to Open Challenges](https://mlsanity.com/paper/2301.10075/) (2023)
- [GATE: A Challenge Set for Gender-Ambiguous Translation Examples](https://mlsanity.com/paper/2303.03975/) (2023)
- [Examining Covert Gender Bias: A Case Study in Turkish and English Machine Translation Models](https://mlsanity.com/paper/2108.10379/) (2021)

## Built on by (most cited first, 4 of 4)

- [Gender Bias in English-to-Greek Machine Translation](https://mlsanity.com/paper/2506.09558/) (2025)
- [Lightweight Fairness for LLM-Based Recommendations via Kernelized Projection and Gated Adapters](https://mlsanity.com/paper/2603.23780/) (2026)
- [Probing Gender Bias in Multilingual LLMs: A Case Study of Stereotypes in Persian](https://mlsanity.com/paper/2509.20168/) (2025)
- [Gender Bias in MT for a Genderless Language: New Benchmarks for Basque](https://mlsanity.com/paper/2603.08153/) (2026)

## Abstract (arXiv)

Neural Machine Translation (NMT) continues to improve in quality and adoption, yet the inadvertent perpetuation of gender bias remains a significant concern. Despite numerous studies on gender bias in translations into English from weakly gendered-languages, there are no benchmarks for evaluating this phenomenon or for assessing mitigation strategies. To address this gap, we introduce GATE X-E, an extension to the GATE (Rarrick et al., 2023) corpus, that consists of human translations from Turkish, Hungarian, Finnish, and Persian into English. Each translation is accompanied by feminine, masculine, and neutral variants. The dataset, which contains between 1250 and 1850 instances for each of the four language pairs, features natural sentences with a wide range of sentence lengths and domains, challenging translation rewriters on various linguistic phenomena. Additionally, we present a translation gender rewriting solution built with GPT-4 and use GATE X-E to evaluate it. We open source our contributions to encourage further research on gender debiasing.