# Attributional Safety Failures in Large Language Models under Code-Mixed Perturbations

arXiv: https://arxiv.org/abs/2505.14469 · HTML: https://mlsanity.com/paper/2505.14469/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security.
- Builds on 19 of its 53 references that are in the map.
- First posted on arXiv 20 May 2025.

## Placed in

- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 19 of 19)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Challenges and Strategies in Cross-Cultural NLP](https://mlsanity.com/paper/2203.10020/) (2022)
- [A Survey of the State of Explainable AI for Natural Language Processing](https://mlsanity.com/paper/2010.00711/) (2020)
- [Detoxifying Language Models Risks Marginalizing Minority Voices](https://mlsanity.com/paper/2104.06390/) (2021)
- [GLUECoS : An Evaluation Benchmark for Code-Switched NLP](https://mlsanity.com/paper/2004.12376/) (2020)
- [A Survey of Code-switched Speech and Language Processing](https://mlsanity.com/paper/1904.00784/) (2019)
- [Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations](https://mlsanity.com/paper/2406.11801/) (2024)
- [How (un)ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queries](https://mlsanity.com/paper/2402.15302/) (2024)
- [Sowing the Wind, Reaping the Whirlwind: The Impact of Editing Language Models](https://mlsanity.com/paper/2401.10647/) (2024)
- [Grammatical Constraints on Intra-sentential Code-Switching: From Theories to Working Models](https://mlsanity.com/paper/1612.04538/) (2016)
- [Soteria: Language-Specific Functional Parameter Steering for Multilingual Safety Alignment](https://mlsanity.com/paper/2502.11244/) (2025)
- [Breaking Boundaries: Investigating the Effects of Model Editing on Cross-linguistic Performance](https://mlsanity.com/paper/2406.11139/) (2024)

## Built on by (most cited first, 6 of 6)

- [AI Safety Training Can be Clinically Harmful](https://mlsanity.com/paper/2604.23445/) (2026)
- [Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global South](https://mlsanity.com/paper/2605.19190/) (2026)
- [Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text in Multilingual Encoders](https://mlsanity.com/paper/2603.19771/) (2026)
- [Lost in Interpretation: The Plausibility-Faithfulness Trade-off in Cross-Lingual Explanations](https://mlsanity.com/paper/2605.19274/) (2026)
- [ArabicDialectSafety: A Dialect-Aware Benchmark for Arabic Content Safety Classification](https://mlsanity.com/paper/2608.01291/) (2026)
- [IndicSafeEval: Safety Robustness of Large Language Models under Multilingual Persuasive Jailbreak Attacks](https://mlsanity.com/paper/2609.03781/) (2026)

## Abstract (arXiv)

While LLMs appear robustly safety-aligned in English, we uncover a catastrophic, overlooked weakness: attributional collapse under code-mixed perturbations. Our systematic evaluation of open models shows that the linguistic camouflage of code-mixing -- ``blending languages within a single conversation'' -- can cause safety guardrails to fail dramatically. Attack success rates (ASR) spike from a benign 9% in monolingual English to 69% under code-mixed inputs, with rates exceeding 90% in non-Western contexts such as Arabic and Hindi. These effects hold not only on controlled synthetic datasets but also on real-world social media traces, revealing a serious risk for billions of users. To explain why this happens, we introduce saliency drift attribution (SDA), an interpretability framework that shows how, under code-mixing, the model's internal attention drifts away from safety-critical tokens (e.g., ``violence'' or ``corruption''), effectively blinding it to harmful intent. Finally, we propose a lightweight translation-based restoration strategy that recovers roughly 80% of the safety lost to code-mixing, offering a practical path toward more equitable and robust LLM safety.