mlsanityData to 7 Oct 2026

Gender Coreference and Bias Evaluation at WMT 2020

arXiv 2010.06018 ↗First posted 12 Oct 2020cs.CLcited by 21

Not on the map. Why?

Abstract (arXiv)

Gender bias in machine translation can manifest when choosing gender inflections based on spurious gender correlations. For example, always translating doctors as men and nurses as women. This can be particularly harmful as models become more popular and deployed within commercial systems. Our work presents the largest evidence for the phenomenon in more than 19 systems submitted to the WMT over four diverse target languages: Czech, German, Polish, and Russian. To achieve this, we use WinoMT, a recent automatic test suite which examines gender coreference and bias when translating from English to languages with grammatical gender. We extend WinoMT to handle two new languages tested in WMT: Polish and Czech. We find that all systems consistently use spurious correlations in the data rather than meaningful contextual information.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 5; 2023: 2; 2024: 4; 2025: 4; 2026: 2.

21 papers.

Sort:

  1. 2021Societal Biases in Language Generation: Progress and Challengescited by 201
  2. 2021Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translationcited by 62
  3. 2022MT-GenEval: A Counterfactual and Contextual Dataset for Evaluating Gender Accuracy in Machine Translationcited by 38
  4. 2021Why don't people use character-level machine translation?cited by 27
  5. 2022The Defeat of the Winograd Schema Challengecited by 25
  6. 2022Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translationcited by 23
  7. 2024What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Studycited by 15
  8. 2023Target-Agnostic Gender-Aware Contrastive Learning for Mitigating Bias in Multilingual Machine Translationcited by 10
  9. 2022What Do Compressed Multilingual Machine Translation Models Forget?cited by 9
  10. 2022Mitigating Gender Bias in Machine Translation through Adversarial Learningcited by 8
  11. 2025Are We Paying Attention to Her? Investigating Gender Disambiguation and Attention in Machine Translationcited by 7
  12. 2024Whose wife is it anyway? Assessing bias against same-gender relationships in machine translationcited by 6
9 more
  1. 2021Amortized Noisy Channel Neural Machine Translationcited by 4
  2. 2025Gender Bias in English-to-Greek Machine Translationcited by 3
  3. 2025Dual Debiasing: Remove Stereotypes and Keep Factual Gender for Fair Language Modeling and Translationcited by 2
  4. 2025What Triggers my Model? Contrastive Explanations Inform Gender Choices by Translation Modelscited by 2
  5. 2024Current State-of-the-Art of Bias Detection and Mitigation in Machine Translation for African and European Languages: a Reviewcited by 1
  6. 2024WMT24 Test Suite: Gender Resolution in Speaker-Listener Dialogue Rolescited by 1
  7. 2023Towards Harmful Erotic Content Detection through Coreference-Driven Contextual Analysiscited by 0
  8. 2026Explaining GAND: A Resource on Gender-Ambiguous Natural Data & Contrastive Attributioncited by 0
  9. 2026Mitigating Gender Bias in English to Romanian Machine Translationcited by 0