mlsanityData to 7 Oct 2026

Mind the Language Gap: Automated and Augmented Evaluation of Bias in LLMs for High- and Low-Resource Languages

arXiv 2504.18560 ↗First posted 19 Apr 2025cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have exhibited impressive natural language processing capabilities but often perpetuate social biases inherent in their training data. To address this, we introduce MultiLingual Augmented Bias Testing (MLA-BiTe), a framework that improves prior bias evaluation methods by enabling systematic multilingual bias testing. MLA-BiTe leverages automated translation and paraphrasing techniques to support comprehensive assessments across diverse linguistic settings. In this study, we evaluate the effectiveness of MLA-BiTe by testing four state-of-the-art LLMs in six languages -- including two low-resource languages -- focusing on seven sensitive categories of discrimination.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 19 Apr 2025.

Builds on

27 of 37 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  6. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  7. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  8. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  9. 2021Ethical and social risks of harm from Language Modelscited by 957
  10. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  11. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  12. 2020Recipes for building an open-domain chatbotcited by 593
15 more
  1. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  2. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  3. 2019Measuring Bias in Contextualized Word Representationscited by 275
  4. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
  5. 2019Evaluating the Underlying Gender Bias in Contextualized Word Embeddingscited by 94
  6. 2023BiasAsker: Measuring the Bias in Conversational AI Systemcited by 52
  7. 2024Social Bias Evaluation for Large Language Models Requires Prompt Variationscited by 39
  8. 2020From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformerscited by 35
  9. 2024A Comprehensive Survey of Bias in LLMs: Current Landscape and Future Directionscited by 27
  10. 2022Using Natural Sentences for Understanding Biases in Language Modelscited by 19
  11. 2024ChatGPT vs Gemini vs LLaMA on Multilingual Sentiment Analysiscited by 13
  12. 2024Social Bias in Large Language Models For Bangla: An Empirical Study on Gender and Religious Biascited by 13
  13. 2024RogueGPT: dis-ethical tuning transforms ChatGPT4 into a Rogue AI in 158 Wordscited by 4
  14. 2024LangBiTe: A Platform for Testing Bias in Large Language Modelscited by 3
  15. 2024Evaluating Large Language Models with fmevalcited by 3

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025MFTCXplain: A Multilingual Benchmark Dataset for Evaluating the Moral Reasoning of LLMs through Multi-hop Hate Speech Explanationcited by 7
  2. 2025AI Diffusion in Low Resource Language Countriescited by 2
  3. 2025Cross-Platform Evaluation of Reasoning Capabilities in Foundation Modelscited by 1
  4. 2025Probing Gender Bias in Multilingual LLMs: A Case Study of Stereotypes in Persiancited by 0