mlsanityData to 14 Oct 2026

Safety Alignment Illusion: The Cross-Lingual Safety Gap in LLMs

arXiv 2608.18131 ↗First posted 26 Jul 2026cs.AI cs.CL cs.CYcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Current safety alignment training for Large Language Models (LLMs) are heavily English-centric. When such safety filters fail for non-English languages, the consequences are immediate and user-facing: voice assistants and spoken dialogue systems may produce stereotype-reinforcing outputs, bypassing the standard English-focused safety alignments and propagating harmful bias to non-English speaking communities. For spoken language technologies deployed across India's linguistically diverse population, this represents a critical failure mode. To address this cross-lingual gap, we introduce INCLUDE (Indian Cultural Lens for Understanding and Detecting Embedded Biases), a multilingual evaluation benchmark designed to quantify Indian-centric socio-cultural biases. INCLUDE consists of 2,604 prompts spanning six prompt languages: English, Hindi, Bengali, Marathi, Tamil, and Hinglish (Hindi-English code-mix). We evaluate ten open- and closed-source LLMs against this benchmark, analyzing 14,988 bias scores. Our statistical results reveal two key findings. First, Bengali yielded the highest average bias score in open-source models. Second, English demonstrated a notable reversal, producing the lowest bias in open-source models but the highest bias in closed-source models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 17 of its 27 references that are in the map. First posted on arXiv 26 Jul 2026.

Builds on

17 of 27 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts23 Jan

Other references · 16

  1. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  2. 2021Re-imagining Algorithmic Fairness in India and Beyondcited by 104
  3. 2022How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions?cited by 92
  4. 2022Re-contextualizing Fairness in NLP: The Case of Indiacited by 55
  5. 2024IndiBias: A Benchmark Dataset to Measure Social Biases in Language Models for Indian Contextcited by 32
  6. 2021Socially Aware Bias Measurements for Hindi Language Representationscited by 29
  7. 2024"They are uncultured": Unveiling Covert Harms and Social Threats in LLM Generated Conversationscited by 25
  8. 2023Indian-BhED: A Dataset for Measuring India-Centric Biases in Large Language Modelscited by 18
  9. 2025How Deep Is Representational Bias in LLMs? The Cases of Caste and Religioncited by 14
  10. 2025FairI Tales: Evaluation of Fairness in Indian Contexts with a Focus on Bias and Stereotypescited by 9
  11. 2025BharatBBQ: A Multilingual Bias Benchmark for Question Answering in the Indian Contextcited by 6
  12. 2019Smart, Responsible, and Upper Caste Only: Measuring Caste Attitudes through Large-Scale Analysis of Matrimonial Profilescited by 5
4 more
  1. 2025Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalitiescited by 5
  2. 2022Cultural Re-contextualization of Fairness Research in Language Technologies in Indiacited by 4
  3. 2025AMBEDKAR-A Multi-level Bias Elimination through a Decoding Approach with Knowledge Augmentation for Robust Constitutional Alignment of Language Modelscited by 2
  4. 2025IndiCASA: A Dataset and Bias Evaluation Framework in LLMs Using Contrastive Embedding Similarity in the Indian Contextcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia
  2. 2026SurakshaEval: An Indic Safety Benchmark for Multilingual LLMs
  3. 2025LinguaSafe: A Comprehensive Multilingual Safety Benchmark for Large Language Models
  4. 2026Bridging the Multilingual Safety Divide: Efficient, Culturally-Aware Alignment for Global South Languages
  5. 2026LLM Safety Alignment in Low-Resource Languages: A Systematic Literature Review
  6. 2025The Hidden Space of Safety: Understanding Preference-Tuned LLMs in Multilingual context
  7. 2025The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It
  8. 2023All Languages Matter: On the Multilingual Safety of Large Language Models