mlsanityData to 14 Oct 2026

Whose wife is it anyway? Assessing bias against same-gender relationships in machine translation

arXiv 2401.04972 ↗First posted 10 Jan 2024cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Machine translation often suffers from biased data and algorithms that can lead to unacceptable errors in system output. While bias in gender norms has been investigated, less is known about whether MT systems encode bias about social relationships, e.g., "the lawyer kissed her wife." We investigate the degree of bias against same-gender relationships in MT systems, using generated template sentences drawn from several noun-gender languages (e.g., Spanish) and comprised of popular occupation nouns. We find that three popular MT services consistently fail to accurately translate sentences concerning relationships between entities of the same gender. The error rate varies considerably based on the context, and same-gender sentences referencing high female-representation occupations are translated with lower accuracy. We provide this work as a case study in the evaluation of intrinsic bias in NLP systems with respect to social relationships.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 27 of its 52 references that are in the map. First posted on arXiv 10 Jan 2024.

Builds on

27 of 52 references.

References · 27

  1. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  2. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  3. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  4. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
  5. 2018Gender Bias in Coreference Resolutioncited by 442
  6. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  7. 2017Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypescited by 253
  8. 2019Evaluating Gender Bias in Machine Translationcited by 249
  9. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  10. 2018Correcting Length Bias in Neural Machine Translationcited by 120
  11. 2021Gender Bias in Machine Translationcited by 117
  12. 2020Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problemcited by 86
15 more
  1. 2019Toward Gender-Inclusive Coreference Resolutioncited by 84
  2. 2021A Survey of Race, Racism, and Anti-Racism in NLPcited by 75
  3. 2021Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in Machine Translationcited by 70
  4. 2020Mitigating Gender Bias in Machine Translation with Target Gender Annotationscited by 42
  5. 2019Simple and Effective Noisy Channel Modeling for Neural Machine Translationcited by 37
  6. 2020Gender Coreference and Bias Evaluation at WMT 2020cited by 21
  7. 2016On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systemscited by 18
  8. 2020Ten Social Dimensions of Conversations and Relationshipscited by 18
  9. 2021Extending Challenge Sets to Uncover Gender Bias in Machine Translation: Impact of Stereotypical Verbs and Adjectivescited by 14
  10. 2019Modeling Fluency and Faithfulness for Diverse Neural Machine Translationcited by 13
  11. 2021Improving Both Domain Robustness and Domain Adaptability in Machine Translationcited by 11
  12. 2022How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contextscited by 10
  13. 2023Target-Agnostic Gender-Aware Contrastive Learning for Mitigating Bias in Multilingual Machine Translationcited by 10
  14. 2020Multilingual Contextual Affective Analysis of LGBT People Portrayals in Wikipediacited by 9
  15. 2021Using Sociolinguistic Variables to Reveal Changing Attitudes Towards Sexuality and Gendercited by 4

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 2.

6 papers.

  1. 2024Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Artcited by 26
  2. 2024Watching the Watchers: Exposing Gender Disparities in Machine Translation Quality Estimationcited by 10
  3. 2025Sports and Women's Sports: Gender Bias in Text Generation with Olympic Datacited by 3
  4. 2024Current State-of-the-Art of Bias Detection and Mitigation in Machine Translation for African and European Languages: a Reviewcited by 1
  5. 2025Theories of "Sexuality" in Natural Language Processing Bias Researchcited by 1
  6. 2024On the Influence of Gender and Race in Romantic Relationship Prediction from Large Language Modelscited by 0