# Gender Coreference and Bias Evaluation at WMT 2020

arXiv: https://arxiv.org/abs/2010.06018 · HTML: https://mlsanity.com/paper/2010.06018/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2020.

## Built on by (most cited first, 21 of 21)

- [Societal Biases in Language Generation: Progress and Challenges](https://mlsanity.com/paper/2105.04054/) (2021)
- [Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation](https://mlsanity.com/paper/2109.03858/) (2021)
- [MT-GenEval: A Counterfactual and Contextual Dataset for Evaluating Gender Accuracy in Machine Translation](https://mlsanity.com/paper/2211.01355/) (2022)
- [Why don't people use character-level machine translation?](https://mlsanity.com/paper/2110.08191/) (2021)
- [The Defeat of the Winograd Schema Challenge](https://mlsanity.com/paper/2201.02387/) (2022)
- [Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation](https://mlsanity.com/paper/2203.09866/) (2022)
- [What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Study](https://mlsanity.com/paper/2410.00545/) (2024)
- [Target-Agnostic Gender-Aware Contrastive Learning for Mitigating Bias in Multilingual Machine Translation](https://mlsanity.com/paper/2305.14016/) (2023)
- [What Do Compressed Multilingual Machine Translation Models Forget?](https://mlsanity.com/paper/2205.10828/) (2022)
- [Mitigating Gender Bias in Machine Translation through Adversarial Learning](https://mlsanity.com/paper/2203.10675/) (2022)
- [Are We Paying Attention to Her? Investigating Gender Disambiguation and Attention in Machine Translation](https://mlsanity.com/paper/2505.08546/) (2025)
- [Whose wife is it anyway? Assessing bias against same-gender relationships in machine translation](https://mlsanity.com/paper/2401.04972/) (2024)
- [Amortized Noisy Channel Neural Machine Translation](https://mlsanity.com/paper/2112.08670/) (2021)
- [Gender Bias in English-to-Greek Machine Translation](https://mlsanity.com/paper/2506.09558/) (2025)
- [Dual Debiasing: Remove Stereotypes and Keep Factual Gender for Fair Language Modeling and Translation](https://mlsanity.com/paper/2501.10150/) (2025)
- [What Triggers my Model? Contrastive Explanations Inform Gender Choices by Translation Models](https://mlsanity.com/paper/2512.08440/) (2025)
- [Current State-of-the-Art of Bias Detection and Mitigation in Machine Translation for African and European Languages: a Review](https://mlsanity.com/paper/2410.21126/) (2024)
- [WMT24 Test Suite: Gender Resolution in Speaker-Listener Dialogue Roles](https://mlsanity.com/paper/2411.06194/) (2024)
- [Towards Harmful Erotic Content Detection through Coreference-Driven Contextual Analysis](https://mlsanity.com/paper/2310.14325/) (2023)
- [Explaining GAND: A Resource on Gender-Ambiguous Natural Data & Contrastive Attribution](https://mlsanity.com/paper/2607.22546/) (2026)
- [Mitigating Gender Bias in English to Romanian Machine Translation](https://mlsanity.com/paper/2608.08606/) (2026)

## Abstract (arXiv)

Gender bias in machine translation can manifest when choosing gender inflections based on spurious gender correlations. For example, always translating doctors as men and nurses as women. This can be particularly harmful as models become more popular and deployed within commercial systems. Our work presents the largest evidence for the phenomenon in more than 19 systems submitted to the WMT over four diverse target languages: Czech, German, Polish, and Russian. To achieve this, we use WinoMT, a recent automatic test suite which examines gender coreference and bias when translating from English to languages with grammatical gender. We extend WinoMT to handle two new languages tested in WMT: Polish and Czech. We find that all systems consistently use spurious correlations in the data rather than meaningful contextual information.