mlsanityData to 14 Oct 2026

Hi Guys or Hi Folks? Benchmarking Gender-Neutral Machine Translation with the GeNTE Corpus

arXiv 2310.05294 ↗First posted 8 Oct 2023cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

Gender inequality is embedded in our communication practices and perpetuated in translation technologies. This becomes particularly apparent when translating into grammatical gender languages, where machine translation (MT) often defaults to masculine and stereotypical representations by making undue binary gender assumptions. Our work addresses the rising demand for inclusive language by focusing head-on on gender-neutral translation from English to Italian. We start from the essentials: proposing a dedicated benchmark and exploring automated evaluation methods. First, we introduce GeNTE, a natural, bilingual test set for gender-neutral translation, whose creation was informed by a survey on the perception and use of neutral language. Based on GeNTE, we then overview existing reference-based evaluation approaches, highlight their limits, and propose a reference-free method more suitable to assess gender-neutral translation.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 33 of its 68 references that are in the map. First posted on arXiv 8 Oct 2023.

Builds on

33 of 68 references.

References · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  3. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  4. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  5. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  6. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  7. 2019Evaluating Gender Bias in Machine Translationcited by 249
  8. 2021Societal Biases in Language Generation: Progress and Challengescited by 202
  9. 2021Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologiescited by 128
  10. 2021Gender Bias in Machine Translationcited by 117
  11. 2019Toward Gender-Inclusive Coreference Resolutioncited by 84
  12. 2019On Measuring Gender Bias in Translation of Gender-neutral Pronounscited by 60
21 more
  1. 2020Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpuscited by 57
  2. 2023"I'm fully who I am": Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generationcited by 56
  3. 2022Welcome to the Modern World of Pronouns: Identity-Inclusive Natural Language Processing beyond Gendercited by 47
  4. 2021NeuTral Rewriter: A Rule-Based and Neural Approach to Automatic Rewriting into Gender-Neutral Alternativescited by 46
  5. 2021They, Them, Theirs: Rewriting with Gender-Neutral Englishcited by 44
  6. 2020Neural Machine Translation Doesn't Translate Gender Coreference Right Unless You Make Itcited by 40
  7. 2022MT-GenEval: A Counterfactual and Contextual Dataset for Evaluating Gender Accuracy in Machine Translationcited by 38
  8. 2023ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languagescited by 34
  9. 2021First the worst: Finding better gender translations during beam searchcited by 29
  10. 2022Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translationcited by 23
  11. 2023Gender Neutralization for an Inclusive Machine Translation: from Theoretical Foundations to Open Challengescited by 22
  12. 2021Investigating Failures of Automatic Translation in the Case of Unambiguous Gendercited by 20
  13. 2023What about em? How Commercial Machine Translation Fails to Handle (Neo-)Pronounscited by 19
  14. 2023GATE: A Challenge Set for Gender-Ambiguous Translation Examplescited by 18
  15. 2021GENder-IT: An Annotated English-Italian Parallel Challenge Set for Cross-Linguistic Natural Gender Phenomenacited by 16
  16. 2022How Conservative are Language Models? Adapting to the Introduction of Gender-Neutral Pronounscited by 16
  17. 2023Exploiting Biased Models to De-bias Text: A Gender-Fair Rewriting Modelcited by 14
  18. 2022User-Centric Gender Rewritingcited by 7
  19. 2023Gender, names and other mysteries: Towards the ambiguous for gender-inclusive translationcited by 7
  20. 2021Improving Gender Translation Accuracy with Filtered Self-Trainingcited by 5
  21. 2023BLEU Meets COMET: Combining Lexical and Neural Metrics Towards Robust Machine Translation Evaluationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 9; 2025: 9; 2026: 1.

20 papers.

Sort:

By area:

  1. 2024Gender Bias in Large Language Models across Multiple Languagescited by 35
  2. 2023A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translationcited by 19
  3. 2024What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Studycited by 15
  4. 2024A Prompt Response to the Demand for Automatic Gender-Neutral Translationcited by 12
  5. 2024Watching the Watchers: Exposing Gender Disparities in Machine Translation Quality Estimationcited by 10
  6. 2024Enhancing Gender-Inclusive Machine Translation with Neomorphemes and Large Language Modelscited by 7
  7. 2025A Multilingual, Culture-First Approach to Addressing Misgendering in LLM Applicationscited by 6
  8. 2024Do language models practice what they preach? Examining language ideologies about gendered language reform encoded in LLMscited by 5
  9. 2025Assumed Identities: Quantifying Gender Bias in Machine Translation of Gender-Ambiguous Occupational Termscited by 5
  10. 2025Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translationcited by 5
  11. 2025GAMBIT+: A Challenge Set for Evaluating Gender Bias in Machine Translation Quality Estimation Metricscited by 4
  12. 2024Generating Gender Alternatives in Machine Translationcited by 3
8 more
  1. 2025Gender-Neutral Machine Translation Strategies in Practicecited by 3
  2. 2025An LLM-as-a-judge Approach for Scalable Gender-Neutral Translation Evaluationcited by 2
  3. 2025Translate With Care: Addressing Gender Bias, Neutrality, and Reasoning in Large Language Model Translationscited by 2
  4. 2024WMT24 Test Suite: Gender Resolution in Speaker-Listener Dialogue Rolescited by 1
  5. 2025GeNRe: A French Gender-Neutral Rewriting System Using Collective Nounscited by 1
  6. 2024Fine-grained Gender Control in Machine Translation with Large Language Modelscited by 0
  7. 2025Gender-Neutral Rewriting in Italian: Models, Approaches, and Trade-offscited by 0
  8. 2026Explaining GAND: A Resource on Gender-Ambiguous Natural Data & Contrastive Attributioncited by 0