mlsanityData to 14 Oct 2026

ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languages

arXiv 2305.10510 ↗First posted 17 May 2023cs.CY cs.AI cs.CLcited by 34

Not on the map. Why?

Abstract (arXiv)

In this multicultural age, language translation is one of the most performed tasks, and it is becoming increasingly AI-moderated and automated. As a novel AI system, ChatGPT claims to be proficient in such translation tasks and in this paper, we put that claim to the test. Specifically, we examine ChatGPT's accuracy in translating between English and languages that exclusively use gender-neutral pronouns. We center this study around Bengali, the 7th most spoken language globally, but also generalize our findings across five other languages: Farsi, Malay, Tagalog, Thai, and Turkish. We find that ChatGPT perpetuates gender defaults and stereotypes assigned to certain occupations (e.g. man = doctor, woman = nurse) or actions (e.g. woman = cook, man = go to work), as it converts gender-neutral pronouns in languages to `he' or `she'. We also observe ChatGPT completely failing to translate the English gender-neutral pronoun `they' into equivalent gender-neutral pronouns in other languages, as it produces translations that are incoherent and incorrect. While it does respect and provide appropriately gender-marked versions of Bengali words when prompted with gender information in English, ChatGPT appears to confer a higher respect to men than to women in the same occupation. We conclude that ChatGPT exhibits the same gender biases which have been demonstrated for tools like Google Translate or MS Translator, as we provide recommendations for a human centered approach for future designers of AIs that perform language translation to better accommodate such low-resource languages.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 35 of its 247 references that are in the map. First posted on arXiv 17 May 2023.

Builds on

35 of 247 references.

References · 35

  1. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  2. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  3. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  4. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  5. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  6. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  7. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 670
  8. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  9. 2017Six Challenges for Neural Machine Translationcited by 335
  10. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 296
  11. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  12. 2019Evaluating Gender Bias in Machine Translationcited by 249
23 more
  1. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 223
  2. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  3. 2018Gender Bias in Neural Natural Language Processingcited by 211
  4. 2021The Values Encoded in Machine Learning Researchcited by 172
  5. 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
  6. 2021Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Modelscited by 141
  7. 2018Assessing Gender Bias in Machine Translation -- A Case Study with Google Translatecited by 121
  8. 2021Empowering News Recommendation with Pre-trained Language Modelscited by 63
  9. 2021Many-to-English Machine Translation Tools, Data, and Pretrained Modelscited by 31
  10. 2020Language Model is All You Need: Natural Language Understanding as Question Answeringcited by 29
  11. 2022Markedness in Visual Semantic AIcited by 29
  12. 2021Self-Supervised Euphemism Detection and Identification for Content Moderationcited by 25
  13. 2022Gender Bias in Word Embeddings: A Comprehensive Analysis of Frequency, Syntax, and Semanticscited by 24
  14. 2018Gender Aware Spoken Language Translation Applied to English-Arabiccited by 18
  15. 2021Transformer-based Language Model Fine-tuning Methods for COVID-19 Fake News Detectioncited by 15
  16. 2021Examining Covert Gender Bias: A Case Study in Turkish and English Machine Translation Modelscited by 15
  17. 2021What Do We Mean by "Accessibility Research"? A Literature Survey of Accessibility Papers in CHI and ASSETS from 1994 to 2019cited by 14
  18. 2021Testing the Generalization of Neural Language Models for COVID-19 Misinformation Detectioncited by 10
  19. 2022Evaluation of Fake News Detection with Knowledge-Enhanced Language Modelscited by 9
  20. 2022Measuring Gender Bias in Word Embeddings of Gendered Languages Requires Disentangling Grammatical Gender Signalscited by 5
  21. 2019The CATS Hackathon: Creating and Refining Test Items for Cybersecurity Concept Inventoriescited by 2
  22. 2019Investigating Crowdsourcing to Generate Distractors for Multiple-Choice Assessmentscited by 1
  23. 2020Experiences and Lessons Learned Creating and Validating Concept Inventories for Cybersecuritycited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 9; 2024: 14; 2025: 5; 2026: 6.

34 papers.

Sort:

By area:

  1. 2023Low-Resource Languages Jailbreak GPT-4cited by 309
  2. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 83
  3. 2023Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Datacited by 66
  4. 2023MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Modelscited by 63
  5. 2024(A)I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal Advicecited by 63
  6. 2024GenderBias-VL: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probingcited by 31
  7. 2023Unveiling Gender Bias in Terms of Profession Across LLMs: Analyzing and Addressing Sociological Implicationscited by 29
  8. 2023Evaluating Biased Attitude Associations of Language Models in an Intersectional Contextcited by 20
  9. 2023Hi Guys or Hi Folks? Benchmarking Gender-Neutral Machine Translation with the GeNTE Corpuscited by 20
  10. 2023TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varietiescited by 16
  11. 2024From Melting Pots to Misrepresentations: Exploring Harms in Generative AIcited by 13
  12. 2025A Comprehensive Analysis of Large Language Model Outputs: Similarity, Diversity, and Biascited by 10
22 more
  1. 2025The Multilingual Divide and Its Impact on Global AI Safetycited by 10
  2. 2024Evaluating and Mitigating Linguistic Discrimination in Large Language Modelscited by 8
  3. 2024'Since Lawyers are Males..': Examining Implicit Gender Bias in Hindi Language Generation by LLMscited by 8
  4. 2023In Generative AI we Trust: Can Chatbots Effectively Verify Political Information?cited by 6
  5. 2024Towards Effective Discrimination Testing for Generative AIcited by 5
  6. 2024Gender Bias in Machine Translation and The Era of Large Language Modelscited by 4
  7. 2023Is the U.S. Legal System Ready for AI's Challenges to Human Values?cited by 2
  8. 2024Transforming Dutch: Debiasing Dutch Coreference Resolution Systems for Non-binary Pronounscited by 2
  9. 2024A Taxonomy of Stereotype Content in Large Language Modelscited by 2
  10. 2024Gender Representation and Bias in Indian Civil Service Mock Interviewscited by 2
  11. 2025Uncertainty Quantification for Evaluating Machine Translation Biascited by 2
  12. 2026How Open Must Language Models be to Enable Reliable Scientific Inference?cited by 2
  13. 2024Modeling the Sacred: Considerations when Using Religious Texts in Natural Language Processingcited by 1
  14. 2024Exploring LGBTQ+ Bias in Generative AI Answers across Different Country and Religious Contextscited by 1
  15. 2025Is Lying Only Sinful in Islam? Exploring Religious Bias in Multilingual Large Language Models Across Major Religionscited by 1
  16. 2026Interrogating Design Homogenization in Web Vibe Codingcited by 1
  17. 2024Misgendering and Assuming Gender in Machine Translation when Working with Low-Resource Languagescited by 0
  18. 2025A Bayesian account of pronoun and neopronoun acquisitioncited by 0
  19. 2026Sima AIunty: Caste Audit in LLM-Driven Matchmakingcited by 0
  20. 2026Decoding-Time Debiasing via Process Reward Models: From Controlled Fill-in to Open-Ended Generationcited by 0
  21. 2026Mod-Guide: An LLM-based Content Moderation Feedback System to Address Insensitive Speech toward Indigenous Ethnic and Religious Minority Communitiescited by 0
  22. 2026Who Receives Respect? A Counterfactual Benchmark for Honorific Allocation Bias in Bengalicited by 0