mlsanityData to 14 Oct 2026

The Rogue Scalpel: Activation Steering Compromises LLM Safety

arXiv 2509.22067 ↗First posted 26 Sep 2025cs.LG cs.AIcited by 21

On the map: LLM safety and security → Safety alignment, refusal and guardrails

also in 1 more
Abstract (arXiv)

Activation steering is a promising technique for controlling LLM behavior by adding semantically meaningful vectors directly into a model's hidden states during inference. It is often framed as a precise, interpretable, and potentially safer alternative to fine-tuning. We demonstrate the opposite: steering systematically breaks model alignment safeguards, making it comply with harmful requests. Through extensive experiments on different model families, we show that even steering in a random direction can increase the probability of harmful compliance from 0% to 1-13%. Alarmingly, steering benign features from a sparse autoencoder (SAE), a common source of interpretable directions, demonstrates a comparable harmful potential. Finally, we show that combining 20 randomly sampled vectors that jailbreak a single prompt creates a universal attack, significantly increasing harmful compliance on unseen requests. These results challenge the paradigm of safety through interpretability, showing that precise control over model internals does not guarantee precise control over model behavior.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 49 of its 60 references that are in the map. First posted on arXiv 26 Sep 2025.

Builds on

49 of 60 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Probing the Robustness of Large Language Models Safety to Latent Perturbations19 Jun
  2. 2024Refusal in Language Models Is Mediated by a Single Direction17 Jun
  3. 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vector18 Apr
  4. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct

Other references · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  9. 2023Let's Verify Step by Stepcited by 3,789
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  11. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  12. 2024A Survey on LLM-as-a-Judgecited by 1,423
33 more
  1. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  2. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  3. 2023Steering Language Models With Activation Engineeringcited by 922
  4. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  5. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  6. 2024Scaling and evaluating sparse autoencoderscited by 553
  7. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
  8. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 451
  9. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  10. 2024Measuring short-form factuality in large language modelscited by 373
  11. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 264
  12. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 218
  13. 2025Open Problems in Mechanistic Interpretabilitycited by 179
  14. 2023Linear Representations of Sentiment in Large Language Modelscited by 163
  15. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 160
  16. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 143
  17. 2024The Prompt Report: A Systematic Survey of Prompt Engineering Techniquescited by 121
  18. 2024What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineeringcited by 90
  19. 2025SAEs Are Good for Steering -- If You Select the Right Featurescited by 78
  20. 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineeringcited by 63
  21. 2025Steering Large Language Model Activations in Sparse Spacescited by 58
  22. 2025Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performancecited by 57
  23. 2024Style Vectors for Steering Generative Large Language Modelcited by 56
  24. 2024A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Modelscited by 52
  25. 2025I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoderscited by 45
  26. 2024Multi-property Steering of Large Language Models with Dynamic Activation Compositioncited by 39
  27. 2025Interpretable Steering of Large Language Models with Feature Guided Activation Additionscited by 35
  28. 2024Extracting Unlearned Information from LLMs with Activation Steeringcited by 21
  29. 2025When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMscited by 10
  30. 2025Probing Latent Subspaces in LLM for AI Security: Identifying and Manipulating Adversarial Statescited by 9
  31. 2025Unintended Misalignment from Agentic Fine-Tuning: Risks and Mitigationcited by 9
  32. 2024Enhancing Multiple Dimensions of Trustworthiness in LLMs via Sparse Activation Controlcited by 5
  33. 2025HAMSA: Hijacking Aligned Compact Models via Stealthy Automationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 20.

21 papers.

Sort:

By area:

  1. 2025Activation Steering for Masked Diffusion Language Modelscited by 9
  2. 2026Steering Externalities: Benign Activation Steering Unintentionally Increases Jailbreak Risk for Large Language Modelscited by 8
  3. 2026Steered LLM Activations are Non-Surjectivecited by 8
  4. 2026Activation Steering for Aligned Open-ended Generation without Sacrificing Coherencecited by 4
  5. 2026MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMscited by 2
  6. 2026Conditional Optimal Bridge for Riemannian Activation Steeringcited by 2
  7. 2026OPIUM: Mitigating Steering Externalities and Over-Refusal via Dual Objective Latent Optimizationcited by 2
  8. 2026ShallowJail: Steering Jailbreaks against Large Language Modelscited by 1
  9. 2026LLM-Guided Prompt Evolution for Password Guessingcited by 1
  10. 2026GeoLAN: Geometric Learning of Latent Explanatory Directions in Large Language Modelscited by 0
  11. 2026Steering grids for sparse-autoencoder features: when a top-context label names an activation regime rather than a causal axiscited by 0
  12. 2026Closing the Activation-Cone Blind Spot: Response-Time Probing and Unified Defensecited by 0
9 more
  1. 2026Safety Cost of Steering Vectors Is Separable and Reduciblecited by 0
  2. 2026Latent Space Refusal Anchoring for Low-Resource African Languages: Mechanistic Safety Recovery Without Retrainingcited by 0
  3. 2026When Safety Speaks a Language: A Mechanistic Analysis of Safety-Language Identity Entanglement in LLMscited by 0
  4. 2026Locating and Steering Refusal Beyond Attentioncited by 0
  5. 2026Training-Free Task Vectors for LLM Behavioral Controlcited by 0
  6. 2026STR: Supervised Transcoder Replacement for Reducing Steering Side Effectscited by 0
  7. 2026Certified Mechanistic Edits: Behavioral Guarantees for Skill Removal and Preservationcited by 0
  8. 2026Extracting Persona Subspaces Through Iterative Nullspace Projection For Modulationcited by 0
  9. 2026Backdooring Sparse Autoencoderscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Analysing the Safety Pitfalls of Steering Vectors
  2. 2025Guiding Giants: Lightweight Controllers for Weighted Activation Steering in LLMs
  3. 2026Steering Vectors are an Adversarial Attack Surface
  4. 2025One-shot Optimized Steering Vectors Mediate Safety-relevant Behaviors in LLMs
  5. 2026Activation Steering Induces Emergent Misalignment: A More Comprehensive Evaluation
  6. 2026Adversarial Robustness of Activation Steering in Large Language Models
  7. 2024SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering
  8. 2026Selective Steering: Norm-Preserving Control Through Discriminative Layer Selection