mlsanityData to 7 Oct 2026

On Measuring Gender Bias in Translation of Gender-neutral Pronouns

arXiv 1905.11684 ↗First posted 28 May 2019cs.CLcited by 59

Not on the map. Why?

Abstract (arXiv)

Ethics regarding social bias has recently thrown striking issues in natural language processing. Especially for gender-related topics, the need for a system that reduces the model bias has grown in areas such as image captioning, content recommendation, and automated employment. However, detection and evaluation of gender bias in the machine translation systems are not yet thoroughly investigated, for the task being cross-lingual and challenging to define. In this paper, we propose a scheme for making up a test set that evaluates the gender bias in a machine translation system, with Korean, a language with gender-neutral pronouns. Three word/phrase sets are primarily constructed, each incorporating positive/negative expressions or occupations; all the terms are gender-independent or at least not biased to one side severely. Then, additional sentence lists are constructed concerning formality of the pronouns and politeness of the sentences. With the generated sentence set of size 4,236 in total, we evaluate gender bias in conventional machine translation systems utilizing the proposed measure, which is termed here as translation gender bias index (TGBI). The corpus and the code for evaluation is available on-line.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 3; 2023: 22; 2024: 15; 2025: 7; 2026: 1.

59 papers.

Sort:

By area:

  1. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
  2. 2021Societal Biases in Language Generation: Progress and Challengescited by 201
  3. 2021Gender Bias in Machine Translationcited by 116
  4. 2021Trustworthy AI: A Computational Perspectivecited by 93
  5. 2021A Survey on Gender Bias in Natural Language Processingcited by 89
  6. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  7. 2022Theories of "Gender" in NLP Bias Researchcited by 62
  8. 2024Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attributioncited by 39
  9. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 27
  10. 2024Identifying Implicit Social Biases in Vision-Language Modelscited by 27
  11. 2021Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue modelscited by 23
  12. 2023Gender Neutralization for an Inclusive Machine Translation: from Theoretical Foundations to Open Challengescited by 22
38 more
  1. 2024LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actionscited by 21
  2. 2023Hi Guys or Hi Folks? Benchmarking Gender-Neutral Machine Translation with the GeNTE Corpuscited by 20
  3. 2021Evaluating Gender Bias in Hindi-English Machine Translationcited by 19
  4. 2023What about em? How Commercial Machine Translation Fails to Handle (Neo-)Pronounscited by 19
  5. 2023A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translationcited by 19
  6. 2023Fairness in Language Models Beyond English: Gaps and Challengescited by 16
  7. 2021Examining Covert Gender Bias: A Case Study in Turkish and English Machine Translation Modelscited by 15
  8. 2023One Wide Feedforward is All You Needcited by 15
  9. 2023''Fifty Shades of Bias'': Normative Ratings of Gender Bias in GPT Generated English Textcited by 15
  10. 2024What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Studycited by 15
  11. 2023Learning Language-Specific Layers for Multilingual Machine Translationcited by 13
  12. 2023CALM : A Multi-task Benchmark for Comprehensive Assessment of Language Model Biascited by 13
  13. 2024Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPTcited by 13
  14. 2023Women Are Beautiful, Men Are Leaders: Gender Stereotypes in Machine Translation and Language Modelingcited by 12
  15. 2024A Prompt Response to the Demand for Automatic Gender-Neutral Translationcited by 12
  16. 2021How to Split: the Effect of Word Segmentation on Gender Bias in Speech Translationcited by 11
  17. 2023Measuring Gender Bias in West Slavic Language Modelscited by 10
  18. 2024Watching the Watchers: Exposing Gender Disparities in Machine Translation Quality Estimationcited by 10
  19. 2022Measuring Harmful Representations in Scandinavian Language Modelscited by 7
  20. 2023Test Suites Task: Evaluation of Gender Fairness in MT with MuST-SHE and INEScited by 6
  21. 2024Silver-Tongued and Sundry: Exploring Intersectional Pronouns with ChatGPTcited by 6
  22. 2021Improving Gender Translation Accuracy with Filtered Self-Trainingcited by 5
  23. 2024MiTTenS: A Dataset for Evaluating Gender Mistranslationcited by 5
  24. 2024Hire Me or Not? Examining Language Model's Behavior with Occupation Attributescited by 5
  25. 2025Assumed Identities: Quantifying Gender Bias in Machine Translation of Gender-Ambiguous Occupational Termscited by 5
  26. 2022Undesirable Biases in NLP: Addressing Challenges of Measurementcited by 4
  27. 2023"I'm" Lost in Translation: Pronoun Missteps in Crowdsourced Data Setscited by 4
  28. 2023Gender Inflected or Bias Inflicted: On Using Grammatical Gender Cues for Bias Evaluation in Machine Translationcited by 4
  29. 2024Gender Bias in Machine Translation and The Era of Large Language Modelscited by 4
  30. 2023Measuring Normative and Descriptive Biases in Language Models Using Census Datacited by 3
  31. 2025Gender Bias in English-to-Greek Machine Translationcited by 3
  32. 2025Gender-Neutral Machine Translation Strategies in Practicecited by 3
  33. 2023How To Build Competitive Multi-gender Speech Translation Models For Controlling Speaker Gender Translationcited by 2
  34. 2024Transforming Dutch: Debiasing Dutch Coreference Resolution Systems for Non-binary Pronounscited by 2
  35. 2025Uncertainty Quantification for Evaluating Machine Translation Biascited by 2
  36. 2025Measuring Gender Bias in Job Title Matching for Grammatical Gender Languagescited by 2
  37. 2025What Triggers my Model? Contrastive Explanations Inform Gender Choices by Translation Modelscited by 2
  38. 2024Beyond Binary Gender: Evaluating Gender-Inclusive Machine Translation with Ambiguous Attitude Wordscited by 1

50 most cited of 59 All 59 (JSON)