mlsanityData to 7 Oct 2026

Refusal Direction is Universal Across Safety-Aligned Languages

arXiv 2505.17306 ↗First posted 22 May 2025cs.CLcited by 37

Not on the map. Why?

Cited by 22 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Refusal mechanisms in large language models (LLMs) are essential for ensuring safety. Recent research has revealed that refusal behavior can be mediated by a single direction in activation space, enabling targeted interventions to bypass refusals. While this is primarily demonstrated in an English-centric context, appropriate refusal behavior is important for any language, but poorly understood. In this paper, we investigate the refusal behavior in LLMs across 14 languages using PolyRefuse, a multilingual safety dataset created by translating malicious and benign English prompts into these languages. We uncover the surprising cross-lingual universality of the refusal direction: a vector extracted from English can bypass refusals in other languages with near-perfect effectiveness, without any additional fine-tuning. Even more remarkably, refusal directions derived from any safety-aligned language transfer seamlessly to others. We attribute this transferability to the parallelism of refusal vectors across languages in the embedding space and identify the underlying mechanism behind cross-lingual jailbreaks. These findings provide actionable insights for building more robust multilingual safety defenses and pave the way for a deeper mechanistic understanding of cross-lingual vulnerabilities in LLMs.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 52 of its 70 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

52 of 70 references.

References · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  8. 2017Deep reinforcement learning from human preferencescited by 3,846
  9. 2016Pointer Sentinel Mixture Modelscited by 2,992
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  11. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  12. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
38 more
  1. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  2. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  3. 2019How multilingual is Multilingual BERT?cited by 642
  4. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  5. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  6. 2017Word Translation Without Parallel Datacited by 480
  7. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  8. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  9. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  10. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  11. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  12. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  13. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  14. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
  15. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  16. 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
  17. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
  18. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 222
  19. 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 165
  20. 2022The Geometry of Multilingual Language Model Representationscited by 94
  21. 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contextscited by 92
  22. 2020On the Language Neutrality of Pre-trained Multilingual Representationscited by 82
  23. 2025Do Multilingual LLMs Think In English?cited by 75
  24. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
  25. 2024Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedbackcited by 64
  26. 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vectorcited by 63
  27. 2020On Learning Universal Representations Across Languagescited by 58
  28. 2025Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languagescited by 57
  29. 2024Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablationcited by 48
  30. 2025Crosslingual Reasoning through Test-Time Scalingcited by 46
  31. 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspectivecited by 44
  32. 2025Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Modelscited by 43
  33. 2024A Cross-Language Investigation into Jailbreak Attacks in Large Language Modelscited by 37
  34. 2024Refusal in LLMs is an Affine Functioncited by 26
  35. 2024mOthello: When Do Cross-Lingual Representation Alignment and Cross-Lingual Transfer Emerge in Multilingual Models?cited by 20
  36. 2021Smoothed Contrastive Learning for Unsupervised Sentence Embeddingcited by 14
  37. 2024Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignmentcited by 14
  38. 2024Robustness of Large Language Models Against Adversarial Attackscited by 12

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 35.

37 papers.

Sort:

By area:

  1. 2025The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating Itcited by 42
  2. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  3. 2025Comparative Analysis of LLM Abliteration Methods: A Cross-Architecture Evaluationcited by 8
  4. 2026From Weights to Activations: Is Steering the Next Frontier of Adaptation?cited by 8
  5. 2026Steering to Say No: Configurable Refusal via Activation Steering in Vision Language Modelscited by 3
  6. 2026LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safetycited by 3
  7. 2026Low-Resource Safety Failures Are Action Failures, Not Representation Failurescited by 2
  8. 2026Where Steering Signals Come From: Activation Source Selection in Activation Steeringcited by 2
  9. 2026Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMscited by 1
  10. 2026Multilingual Safety Alignment via Self-Distillationcited by 1
  11. 2026Unlearners Can Lie: Evaluating and Improving Honesty in LLM Unlearningcited by 1
  12. 2026The Illusion of Cross-Lingual Safety in Low-Resource Languagescited by 1
25 more
  1. 2026BabelSteering: Multilingual Safety Alignment via English Steering Vectorscited by 1
  2. 2026Linear Script Representations in Speech Foundation Models Enable Zero-Shot Transliterationcited by 0
  3. 2026SteerEval: Inference-time Interventions Strengthen Multilingual Generalization in Neural Summarization Metricscited by 0
  4. 2026Safe Transformer: An Explicit Safety Bit For Interpretable And Controllable Alignmentcited by 0
  5. 2026C-Mining: Unsupervised Discovery of Seeds for Cultural Data Synthesis via Geometric Misalignmentcited by 0
  6. 2026Why Do Safety Guardrails Degrade Across Languages?cited by 0
  7. 2026Residual Paving: Diagnosing the Routing Bottleneck in Selective Refusal Editingcited by 0
  8. 2026Understanding LLM Behavior in Multi-Target Cross-Lingual Summarizationcited by 0
  9. 2026Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attackcited by 0
  10. 2026LoMC: Localized Multidirectional Correction for Refusal Suppression in Routed Foundation Modelscited by 0
  11. 2026Minionese: Comprehensive Benchmark and Mechanistic Study of Multilingual LLM Safetycited by 0
  12. 2026Decided Upstream, Written Late: Locating and Pricing the Cross-Lingual Refusal Circuit of a Multilingual MoEcited by 0
  13. 2026LLM Safety Alignment in Low-Resource Languages: A Systematic Literature Reviewcited by 0
  14. 2026Shared Circuits for Shared Grammar: Tracing Subject-Verb Agreement Across Languagescited by 0
  15. 2026Who Pays More for Safety? Measuring the Disparate Cost of Safety Alignment across Languagescited by 0
  16. 2026Linear Probing Provides Robust and Efficient Detection of Machine-Generated Textcited by 0
  17. 2026When Safety Speaks a Language: A Mechanistic Analysis of Safety-Language Identity Entanglement in LLMscited by 0
  18. 2026Recurrence Is Not Enough: Causally Validating Multilingual SAE Translation Features in Gemma 2 and 3cited by 0
  19. 2026Semantic Fibers and Cross-Gram Interference: A Calculus of Safety Drift in Overcomplete Representationscited by 0
  20. 2026Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliterationcited by 0
  21. 2026From Trait Vectors to Circuits: Tracing Refusal and Sycophancy Through Language Modelscited by 0
  22. 2026Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidancecited by 0
  23. 2026Hard Negatives Reveal What Easy Negatives Hide: Cross-Lingual Harmfulness Representations Degrade with Resource Tier Under Hard Negativescited by 0
  24. 2026CG-Probes: Recovering Guardrail Directions from Patient Query Embeddingscited by 0
  25. 2026Tool Mediation Alters Refusal Mechanisms in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Localizing Safety Alignment: MLP Layers and Mid-Network Blocks Encode Refusal Behavior in Large Language Models
  2. 2026Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment
  3. 2026Multilingual Refusal Alignment for Safer Large Language Models
  4. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence
  5. 2026Latent-space Attacks for Refusal Evasion in Language Models
  6. 2026LSR: Linguistic Safety Robustness Benchmark for Low-Resource West African Languages
  7. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Models
  8. 2026The Geometry of Refusal: Linear Instability in Safety-Aligned LLMs