mlsanityData to 7 Oct 2026

Soteria: Language-Specific Functional Parameter Steering for Multilingual Safety Alignment

arXiv 2502.11244 ↗First posted 16 Feb 2025cs.CL cs.AIcited by 10

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Ensuring consistent safety across multiple languages remains a significant challenge for large language models (LLMs). We introduce Soteria, a lightweight yet powerful strategy that locates and minimally adjusts the "functional heads" most responsible for harmful content generation in each language. By altering only a fraction of parameters, Soteria drastically reduces policy violations without sacrificing overall model performance, even in low-resource settings. To rigorously evaluate our approach, we also present XThreatBench, a specialized multilingual dataset capturing fine-grained harmful behaviors drawn from real policy guidelines. Experiments with leading open-source LLMs (e.g., Llama, Qwen, Mistral) show that Soteria consistently improves safety metrics across high-, mid-, and low-resource languages. These findings highlight a promising path toward scalable, linguistically attuned, and ethically aligned LLMs worldwide.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 38 of its 73 references that are in the map. First posted on arXiv 16 Feb 2025.

Builds on

38 of 73 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations17 Jun
  2. 2024Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic19 Feb
  3. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct

Other references · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  8. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  9. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  10. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  11. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  12. 2019Are Sixteen Heads Really Better than One?cited by 803
23 more
  1. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  2. 2021Causal Abstractions of Neural Networkscited by 401
  3. 2017Mitigating Adversarial Effects Through Randomizationcited by 382
  4. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  5. 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
  6. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  7. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 207
  8. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  9. 2024How do Large Language Models Handle Multilingualism?cited by 148
  10. 2023Tokenizer Choice For LLM Training: Negligible or Crucial?cited by 105
  11. 2023Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMscited by 103
  12. 2024On scalable oversight with weak LLMs judging strong LLMscited by 94
  13. 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
  14. 2024LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedbackcited by 51
  15. 2023Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Promptscited by 45
  16. 2024How (un)ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queriescited by 26
  17. 2022Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resourcescited by 25
  18. 2024Multilingual Large Language Models and Curse of Multilingualitycited by 13
  19. 2024Learn and Unlearn: Addressing Misinformation in Multilingual LLMscited by 11
  20. 2024Accelerating Multilingual Language Model for Excessively Tokenized Languagescited by 8
  21. 2024How Multilingual Are Large Language Models Fine-Tuned for Translation?cited by 6
  22. 2024RITFIS: Robust input testing framework for LLMs-based intelligent softwarecited by 3
  23. 2025Turning Logic Against Itself : Probing Model Defenses Through Contrastive Questionscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 8.

10 papers.

  1. 2025Attributional Safety Failures in Large Language Models under Code-Mixed Perturbationscited by 6
  2. 2025AURA: Affordance-Understanding and Risk-aware Alignment Technique for Large Language Modelscited by 2
  3. 2026Low-Resource Safety Failures Are Action Failures, Not Representation Failurescited by 2
  4. 2026Bye Bye Perspective API: Lessons for Building and Governing Measurement Infrastructurecited by 1
  5. 2026The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languagescited by 1
  6. 2026BabelSteering: Multilingual Safety Alignment via English Steering Vectorscited by 1
  7. 2026Attributing and Exploiting Safety Vectors through Global Optimization in Large Language Modelscited by 0
  8. 2026Lost in Interpretation: The Plausibility-Faithfulness Trade-off in Cross-Lingual Explanationscited by 0
  9. 2026LLM Safety Alignment in Low-Resource Languages: A Systematic Literature Reviewcited by 0
  10. 2026Who Pays More for Safety? Measuring the Disparate Cost of Safety Alignment across Languagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026PolicyAlign: Direct Policy-Based Safety Alignment for Large Language Models
  2. 2025The Hidden Space of Safety: Understanding Preference-Tuned LLMs in Multilingual context
  3. 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts
  4. 2026SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment
  5. 2023All Languages Matter: On the Multilingual Safety of Large Language Models
  6. 2024Towards Comprehensive Post Safety Alignment of Large Language Models via Safety Patching
  7. 2026Can Safety Emerge from Weak Supervision? A Systematic Analysis of Small Language Models
  8. 2026When Safety Speaks a Language: A Mechanistic Analysis of Safety-Language Identity Entanglement in LLMs