mlsanityData to 7 Oct 2026

SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signals

arXiv 2502.01042 ↗First posted 3 Feb 2025cs.LGcited by 29

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Large language models (LLMs) exhibit exceptional capabilities across various tasks but also pose risks by generating harmful content. Existing safety mechanisms, while improving model safety, often lead to overly cautious behavior and fail to fully leverage LLMs' internal cognitive processes. Inspired by humans' reflective thinking capability, we first show that LLMs can similarly perform internal assessments about safety in their internal states. Building on this insight, we propose SafeSwitch, a dynamic framework that regulates unsafe outputs by utilizing the prober-based internal state monitor that actively detects harmful intentions, and activates a safety head that leads to safer and more conservative responses only when necessary. SafeSwitch reduces harmful outputs by approximately 80% on harmful queries while maintaining strong utility, reaching a Pareto optimal among several methods. Our method is also advantageous over traditional methods in offering more informative, context-aware refusals, and achieves these benefits while only tuning less than 6% of the original parameters. SafeSwitch demonstrates large language models' capacity for self-awareness and reflection regarding safety, offering a promising approach to more nuanced and effective safety controls. Codes for this work are available at https://github.com/Hanpx20/SafeSwitch.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 33 of its 50 references that are in the map. First posted on arXiv 3 Feb 2025.

Builds on

33 of 50 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Refusal in Language Models Is Mediated by a Single Direction17 Jun
  2. 2024Improving Alignment and Robustness with Circuit Breakers6 Jun
  3. 2024Representation Noising: A Defence Mechanism Against Harmful Finetuning23 May
  4. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct

Other references · 29

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2023Let's Verify Step by Stepcited by 3,598
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  9. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  10. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
  11. 2023Large Language Modelscited by 774
  12. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
17 more
  1. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  2. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  3. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  4. 2023Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraintcited by 346
  5. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 308
  6. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  7. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
  8. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  9. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 176
  10. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 174
  11. 2024Rule Based Rewards for Language Model Safetycited by 106
  12. 2024LLM Internal States Reveal Hallucination Risk Faced With a Querycited by 80
  13. 2023Word Embeddings Are Steers for Language Modelscited by 70
  14. 2024Tradeoffs Between Alignment and Helpfulness in Language Models with Steering Methodscited by 26
  15. 2024Why Does New Knowledge Create Messy Ripple Effects in LLMs?cited by 18
  16. 2024EVEDIT: Event-based Knowledge Editing with Deductive Editing Boundariescited by 16
  17. 2024FactCheckmate: Preemptively Detecting and Mitigating Hallucinations in LMscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 20.

29 papers.

Sort:

By area:

  1. 2025LLMs Encode Harmfulness and Refusal Separatelycited by 75
  2. 2025Self-Aligned Reward: Towards Effective and Efficient Reasonerscited by 7
  3. 2025Towards Understanding and Improving Refusal in Compressed Models via Mechanistic Interpretabilitycited by 6
  4. 2025Training-Free Policy Violation Detection via Activation-Space Whitening in LLMscited by 6
  5. 2026GAVEL: Towards Rule-Based Safety Through Activation Monitoringcited by 4
  6. 2026Steering at the Source: Style Modulation Heads for Robust Persona Controlcited by 4
  7. 2025Context Engineering for Trustworthiness: Rescorla Wagner Steering Under Mixed and Inappropriate Contextscited by 3
  8. 2025The Narrative Continuity Test: A Conceptual Framework for Evaluating Identity Persistence in AI Systemscited by 3
  9. 2026Latent Structure of Affective Representations in Large Language Modelscited by 3
  10. 2026Before the Last Token: Diagnosing Final-Token Safety Probe Failurescited by 3
  11. 2026Investigating and Alleviating Harm Amplification in LLM Interactionscited by 2
  12. 2025Graph-Regularized Sparse Autoencoders for LLM Safety Steeringcited by 1
17 more
  1. 2026Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMscited by 1
  2. 2026Chain of Risk: Safety Failures in Large Reasoning Models and Mitigation via Adaptive Multi-Principle Steeringcited by 1
  3. 2026Paved with True Intents: Intent-Aware Training Improves LLM Safety Classification Across Training Regimescited by 1
  4. 2025Mass-Scale Analysis of In-the-Wild Conversations Reveals Complexity Bounds on LLM Jailbreakingcited by 0
  5. 2025SafeConstellations: Mitigating Over-Refusals in LLMs Through Task-Aware Representation Steeringcited by 0
  6. 2026Light Alignment Improves LLM Safety via Model Self-Reflection with a Single Neuroncited by 0
  7. 2026Learning to Stay Safe: Adaptive Regularization Against Safety Degradation during Fine-Tuningcited by 0
  8. 2026Safe Transformer: An Explicit Safety Bit For Interpretable And Controllable Alignmentcited by 0
  9. 2026Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Modelscited by 0
  10. 2026kNNGuard: Turning LLM Hidden Activations into a Training-Free Configurable Guardrailcited by 0
  11. 2026Breaking Refusal in the First Half: A Mechanistic Study of the Prefill Jailbreakcited by 0
  12. 2026Probabilistic Concept-Aware Steering for Trustworthy LLM Inferencecited by 0
  13. 2026Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agentscited by 0
  14. 2026Recognition-Refusal Misalignment in LLMs: Why Models Answer Structurally Unanswerable Questionscited by 0
  15. 2026Locating and Steering Refusal Beyond Attentioncited by 0
  16. 2026First Token Matters: Understanding Safety Collapse in Large Reasoning Modelscited by 0
  17. 2026SafeTune: A Unified Faithful Library for Auditing and Repairing Safety Drift in Fine-Tuned LLMscited by 0