mlsanityData to 7 Oct 2026

When Safety Speaks a Language: A Mechanistic Analysis of Safety-Language Identity Entanglement in LLMs

arXiv 2608.29936 ↗First posted 30 Aug 2026cs.CL cs.LGcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Safety alignment of large language models (LLMs) degrades across languages, yet the internal mechanism driving this asymmetry remains poorly understood. Our work, therefore, presents a systematic mechanistic analysis of multilingual safety using sparse autoencoder (SAE) features, sparse interpretable directions in the residual stream associated with harmful and harmless model behavior across three instruction-tuned LLMs, eight languages, and all model layers. We observe that safety-relevant features are architecture-dependent in terms of where they are located and how they are distributed across layers. Additionally, they are geometrically entangled with language identity and exhibit cross-lingual sharing patterns, i.e., languages share safety features to varying degrees across model depths and architectures. This safety-language entanglement has direct consequences such that ablating safety features impacts not only harmful response rates but also target language, with the degree of intervention predicted by the relationship between safety and language features. Our findings qualify the language-universality of safety alignment as architecture-dependent and offer a mechanistic account of multilingual safety interventions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 31 of its 43 references that are in the map. First posted on arXiv 30 Aug 2026.

Builds on

31 of 43 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026Multilingual Safety Alignment Via Sparse Weight Editing26 Feb
  2. 2026Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment18 Feb
  3. 2025Response-Based Knowledge Distillation for Multilingual Jailbreak Prevention Unwittingly Compromises Safety8 Dec
  4. 2025MPO: Multilingual Safety Alignment via Reward Gap Optimization22 May

Other references · 27

  1. 2016Pointer Sentinel Mixture Modelscited by 2,992
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  4. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  5. 2016Bag of Tricks for Efficient Text Classificationcited by 869
  6. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  7. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  8. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 283
  9. 2024Understanding and Mitigating Language Confusion in LLMscited by 75
  10. 2025I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoderscited by 43
  11. 2025The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating Itcited by 42
  12. 2025Refusal Direction is Universal Across Safety-Aligned Languagescited by 37
15 more
  1. 2024How (un)ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queriescited by 26
  2. 2025Language Mixing in Reasoning Language Models: Patterns, Impact, and Internal Causescited by 20
  3. 2025The Rogue Scalpel: Activation Steering Compromises LLM Safetycited by 18
  4. 2025Beyond I'm Sorry, I Can't: Dissecting Large Language Model Refusalcited by 17
  5. 2025LinguaSafe: A Comprehensive Multilingual Safety Benchmark for Large Language Modelscited by 13
  6. 2025Breaking Bad Tokens: Detoxification of LLMs Using Sparse Autoencoderscited by 8
  7. 2026Who Transfers Safety? Identifying and Targeting Cross-Lingual Shared Safety Neuronscited by 7
  8. 2025SAFER: Probing Safety in Reward Models with Sparse Autoencodercited by 6
  9. 2025How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoderscited by 5
  10. 2025Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Frameworkcited by 5
  11. 2025Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languagescited by 4
  12. 2026Sparse Autoencoders are Capable LLM Jailbreak Mitigatorscited by 4
  13. 2025Evaluating Sparse Autoencoders for Monosemantic Representationcited by 3
  14. 2025Uncovering Cross-Linguistic Disparities in LLMs using Sparse Autoencoderscited by 1
  15. 2025Graph-Regularized Sparse Autoencoders for LLM Safety Steeringcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Multilingual Safety Signals Are Multi-Layered: Filtering Safety-Degrading Data for Safer LLMs
  2. 2026The Illusion of Cross-Lingual Safety in Low-Resource Languages
  3. 2024Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons
  4. 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts
  5. 2025The Hidden Space of Safety: Understanding Preference-Tuned LLMs in Multilingual context
  6. 2026BabelSteering: Multilingual Safety Alignment via English Steering Vectors
  7. 2024Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models
  8. 2026LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safety