mlsanityData to 14 Oct 2026

Attributional Safety Failures in Large Language Models under Code-Mixed Perturbations

arXiv 2505.14469 ↗First posted 20 May 2025cs.CL cs.AIcited by 6

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

While LLMs appear robustly safety-aligned in English, we uncover a catastrophic, overlooked weakness: attributional collapse under code-mixed perturbations. Our systematic evaluation of open models shows that the linguistic camouflage of code-mixing -- ``blending languages within a single conversation'' -- can cause safety guardrails to fail dramatically. Attack success rates (ASR) spike from a benign 9% in monolingual English to 69% under code-mixed inputs, with rates exceeding 90% in non-Western contexts such as Arabic and Hindi. These effects hold not only on controlled synthetic datasets but also on real-world social media traces, revealing a serious risk for billions of users. To explain why this happens, we introduce saliency drift attribution (SDA), an interpretability framework that shows how, under code-mixing, the model's internal attention drifts away from safety-critical tokens (e.g., ``violence'' or ``corruption''), effectively blinding it to harmful intent. Finally, we propose a lightweight translation-based restoration strategy that recovers roughly 80% of the safety lost to code-mixing, offering a practical path toward more equitable and robust LLM safety.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 19 of its 53 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

19 of 53 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Soteria: Language-Specific Functional Parameter Steering for Multilingual Safety Alignment16 Feb
  2. 2024Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations17 Jun
  3. 2024How (un)ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queries23 Feb

Other references · 16

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  4. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  5. 2021Ethical and social risks of harm from Language Modelscited by 966
  6. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
  7. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  8. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  9. 2022Challenges and Strategies in Cross-Cultural NLPcited by 218
  10. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  11. 2021Detoxifying Language Models Risks Marginalizing Minority Voicescited by 107
  12. 2020GLUECoS : An Evaluation Benchmark for Code-Switched NLPcited by 102
4 more
  1. 2019A Survey of Code-switched Speech and Language Processingcited by 75
  2. 2024Sowing the Wind, Reaping the Whirlwind: The Impact of Editing Language Modelscited by 20
  3. 2016Grammatical Constraints on Intra-sentential Code-Switching: From Theories to Working Modelscited by 10
  4. 2024Breaking Boundaries: Investigating the Effects of Model Editing on Cross-linguistic Performancecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Lost in Translation? A Comparative Study on the Cross-Lingual Transfer of Composite Harms
  2. 2026Contrastive Attribution in the Wild: An Interpretability Analysis of LLM Failures on Realistic Benchmarks
  3. 2025Safe in the Future, Dangerous in the Past: Dissecting Temporal and Linguistic Vulnerabilities in LLMs
  4. 2026Beyond Unsafe Detection: Counterfactually Anchored Evidence Attribution for Multi-Turn LLM Safety Failures
  5. 2026TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety
  6. 2026When Attribution Patching Lies: Diagnosis and a Second-Order Correction
  7. 2025Exploring the Secondary Risks of Large Language Models
  8. 2026Why Do Safety Guardrails Degrade Across Languages?