Translate With Care: Addressing Gender Bias, Neutrality, and Reasoning in Large Language Model Translations
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Addressing gender bias and maintaining logical coherence in machine translation remains challenging, particularly when translating between natural gender languages, like English, and genderless languages, such as Persian, Indonesian, and Finnish. We introduce the Translate-with-Care (TWC) dataset, comprising 3,950 challenging scenarios across six low- to mid-resource languages, to assess translation systems' performance. Our analysis of diverse technologies, including GPT-4, mBART-50, NLLB-200, and Google Translate, reveals a universal struggle in translating genderless content, resulting in gender stereotyping and reasoning errors. All models preferred masculine pronouns when gender stereotypes could influence choices. Google Translate and GPT-4 showed particularly strong bias, favoring male pronouns 4-6 times more than feminine ones in leadership and professional success contexts. Fine-tuning mBART-50 on TWC substantially resolved these biases and errors, led to strong generalization, and surpassed proprietary LLMs while remaining open-source. This work emphasizes the need for targeted approaches to gender and semantic coherence in machine translation, particularly for genderless languages, contributing to more equitable and accurate translation systems.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 14 of its 59 references that are in the map. First posted on arXiv 31 May 2025.
Builds on
14 of 59 references.
References · 14
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
- 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
- 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
- 2023Seamless: Multilingual Expressive and Streaming Speech Translationcited by 192
- 2023Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translationcited by 74
- 2020The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Researchcited by 67
- 2023BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languagescited by 60
- 2023BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Modelscited by 42
- 2023Fine-tuning Large Language Models for Adaptive Machine Translationcited by 21
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025An LLM-as-a-judge Approach for Scalable Gender-Neutral Translation Evaluation
- 2023Women Are Beautiful, Men Are Leaders: Gender Stereotypes in Machine Translation and Language Modeling
- 2025EuroGEST: Investigating gender stereotypes in multilingual language models
- 2025Probing Gender Bias in Multilingual LLMs: A Case Study of Stereotypes in Persian
- 2025Evaluating Machine Translation Datasets for Low-Web Data Languages: A Gendered Lens
- 2024Significance of Chain of Thought in Gender Bias Mitigation for English-Dravidian Machine Translation
- 2026FairQE: Multi-Agent Framework for Mitigating Gender Bias in Translation Quality Estimation
- 2021Gender Bias Amplification During Speed-Quality Optimization in Neural Machine Translation