Cross-Lingual Bias in Large Language Models: A Comparative Analysis of English and Swahili
On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment
Abstract (arXiv)
Large language models are increasingly deployed in multilingual contexts, yet safety alignment and bias evaluation remain overwhelmingly English-centric. We investigate whether social biases generalise across languages by submitting 4,900 symmetric English--Swahili prompt pairs to GPT-5.2 and Gemini 2.5 Flash across nine demographic bias axes, yielding 19,600 completions evaluated for stereotype prevalence, sentiment, refusal behaviour, and cross-lingual semantic similarity. Our findings show that bias transforms rather than transfers: stereotype rates shifted by up to 12 percentage points on specific axes, Gemini's neutral-sentiment rate doubled in Swahili, and GPT-5.2 refused 169 prompts in English and zero in Swahili, consistent with refusal behaviour anchored to English-language surface forms at the behavioural level. Over 55% of prompt pairs produced semantically dissimilar completions across both models. These reinforce the idea that English-only bias audits do not produce adequate coverage for multilingual deployment.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 12 of its 24 references that are in the map. First posted on arXiv 4 Aug 2026.
Builds on
12 of 24 references.
References · 12
- 2017Attention Is All You Needcited by 59,932
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
- 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
- 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
- 2023Low-Resource Languages Jailbreak GPT-4cited by 290
- 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
- 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 82
- 2024MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMscited by 32
- 2026LinguaMap: Which Layers of LLMs Speak Your Language and How to Tune Them?cited by 3
- 2025A Cross-Lingual Analysis of Bias in Large Language Models Using Romanian Historycited by 1
- 2025Cross-Language Bias Examination in Large Language Modelscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Assessing Agentic Large Language Models in Multilingual National Bias
- 2023What Do Llamas Really Think? Revealing Preference Biases in Language Model Representations
- 2025Surfacing Subtle Stereotypes: A Multilingual, Debate-Oriented Evaluation of Modern LLMs
- 2025Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMs
- 2024Bias and Volatility: A Statistical Framework for Evaluating Large Language Model's Stereotypes and the Associated Generation Inconsistency
- 2026Bias in the Ear of the Listener: Assessing Sensitivity in Audio Language Models Across Linguistic, Demographic, and Positional Variations
- 2025Mitigating Social Bias in English and Urdu Language Models Using PRM-Guided Candidate Selection and Sequential Refinement
- 2023SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models