Attributional Safety Failures in Large Language Models under Code-Mixed Perturbations
On the map: LLM safety and security → Safety alignment, refusal and guardrails
Abstract (arXiv)
While LLMs appear robustly safety-aligned in English, we uncover a catastrophic, overlooked weakness: attributional collapse under code-mixed perturbations. Our systematic evaluation of open models shows that the linguistic camouflage of code-mixing -- ``blending languages within a single conversation'' -- can cause safety guardrails to fail dramatically. Attack success rates (ASR) spike from a benign 9% in monolingual English to 69% under code-mixed inputs, with rates exceeding 90% in non-Western contexts such as Arabic and Hindi. These effects hold not only on controlled synthetic datasets but also on real-world social media traces, revealing a serious risk for billions of users. To explain why this happens, we introduce saliency drift attribution (SDA), an interpretability framework that shows how, under code-mixing, the model's internal attention drifts away from safety-critical tokens (e.g., ``violence'' or ``corruption''), effectively blinding it to harmful intent. Finally, we propose a lightweight translation-based restoration strategy that recovers roughly 80% of the safety lost to code-mixing, offering a practical path toward more equitable and robust LLM safety.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 19 of its 53 references that are in the map. First posted on arXiv 20 May 2025.
Builds on
19 of 53 references.
Closest ancestors · 3
Same direction, newest first
- 2025Soteria: Language-Specific Functional Parameter Steering for Multilingual Safety Alignment16 Feb
- 2024Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations17 Jun
- 2024How (un)ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queries23 Feb
Other references · 16
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
- 2021Ethical and social risks of harm from Language Modelscited by 966
- 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
- 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
- 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
- 2022Challenges and Strategies in Cross-Cultural NLPcited by 218
- 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
- 2021Detoxifying Language Models Risks Marginalizing Minority Voicescited by 107
- 2020GLUECoS : An Evaluation Benchmark for Code-Switched NLPcited by 102
4 more
- 2019A Survey of Code-switched Speech and Language Processingcited by 75
- 2024Sowing the Wind, Reaping the Whirlwind: The Impact of Editing Language Modelscited by 20
- 2016Grammatical Constraints on Intra-sentential Code-Switching: From Theories to Working Modelscited by 10
- 2024Breaking Boundaries: Investigating the Effects of Model Editing on Cross-linguistic Performancecited by 4
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026AI Safety Training Can be Clinically Harmfulcited by 1
- 2026Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global Southcited by 1
- 2026Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text in Multilingual Encoderscited by 0
- 2026Lost in Interpretation: The Plausibility-Faithfulness Trade-off in Cross-Lingual Explanationscited by 0
- 2026ArabicDialectSafety: A Dialect-Aware Benchmark for Arabic Content Safety Classificationcited by 0
- 2026IndicSafeEval: Safety Robustness of Large Language Models under Multilingual Persuasive Jailbreak Attackscited by 0
- 2026IndicSafeEval: Safety Robustness of Large Language Models under Multilingual Persuasive Jailbreak Attacks3 Sep 2026
- 2026ArabicDialectSafety: A Dialect-Aware Benchmark for Arabic Content Safety Classification2 Aug 2026
- 2026Lost in Interpretation: The Plausibility-Faithfulness Trade-off in Cross-Lingual Explanations19 May 2026
- 2026Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global South18 May 2026
- 2026AI Safety Training Can be Clinically Harmful25 Apr 2026
- 2026Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text in Multilingual Encoders20 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026Lost in Translation? A Comparative Study on the Cross-Lingual Transfer of Composite Harms
- 2026Contrastive Attribution in the Wild: An Interpretability Analysis of LLM Failures on Realistic Benchmarks
- 2025Safe in the Future, Dangerous in the Past: Dissecting Temporal and Linguistic Vulnerabilities in LLMs
- 2026Beyond Unsafe Detection: Counterfactually Anchored Evidence Attribution for Multi-Turn LLM Safety Failures
- 2026TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety
- 2026When Attribution Patching Lies: Diagnosis and a Second-Order Correction
- 2025Exploring the Secondary Risks of Large Language Models
- 2026Why Do Safety Guardrails Degrade Across Languages?