mlsanityData to 7 Oct 2026

Analysing the Safety Pitfalls of Steering Vectors

arXiv 2603.24543 ↗First posted 25 Mar 2026cs.CR cs.CLcited by 9

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Activation steering has emerged as a powerful tool to shape LLM behavior without the need for weight updates. While its inherent brittleness and unreliability are well-documented, its safety implications remain underexplored. In this work, we present a systematic safety audit of steering vectors obtained with Contrastive Activation Addition (CAA), a widely used steering approach, under a unified evaluation protocol. Using JailbreakBench as benchmark, we show that steering vectors consistently influence the success rate of jailbreak attacks, with stronger amplification under simple template-based attacks. Across LLM families and sizes, steering the model in specific directions can drastically increase (up to 57%) or decrease (up to 50%) its attack success rate (ASR), depending on the targeted behavior. We attribute this phenomenon to the overlap between the steering vectors and the latent directions of refusal behavior. Thus, we offer a traceable explanation for this discovery. Together, our findings reveal the previously unobserved origin of this safety gap in LLMs, highlighting a trade-off between controllability and safety.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 40 of its 54 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

40 of 54 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence24 Feb
  2. 2025The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions13 Feb
  3. 2024Refusal in Language Models Is Mediated by a Single Direction17 Jun
  4. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deep10 Jun
  5. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications7 Feb

Other references · 35

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  9. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  10. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  11. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  12. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
23 more
  1. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  2. 2023Steering Language Models With Activation Engineeringcited by 824
  3. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  4. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  5. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 351
  6. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  7. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
  8. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  9. 2023Localizing Model Behavior with Path Patchingcited by 182
  10. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
  11. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
  12. 2024Improving Steering Vectors by Targeting Sparse Autoencoder Featurescited by 86
  13. 2023Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMscited by 82
  14. 2024Language Models Resist Alignment: Evidence From Data Compressioncited by 50
  15. 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspectivecited by 44
  16. 2025Understanding (Un)Reliability of Steering Vectors in Language Modelscited by 39
  17. 2024Extending Activation Steering to Broad Skills and Multiple Behaviourscited by 36
  18. 2025Steering off Course: Reliability Challenges in Steering Language Modelscited by 29
  19. 2024Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Modelscited by 23
  20. 2024Comparing Bottom-Up and Top-Down Steering Approaches on In-Context Learning Taskscited by 13
  21. 2025Improving LLM Reasoning through Interpretable Role-Playing Steeringcited by 11
  22. 2025Aligning to What? Limits to RLHF Based Alignmentcited by 8
  23. 2024Defending Large Language Models Against Attacks With Residual Stream Activation Analysiscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026Persona Non Grata: Single-Method Safety Evaluation Is Incomplete for Persona-Imbued LLMscited by 1
  2. 2026Dual-Stance Evaluation of Sycophancy: The Structure of Agreement and the Limits of Interventioncited by 1
  3. 2026Forecasting Side Effects of Activation Steeringcited by 1
  4. 2026Safety Cost of Steering Vectors Is Separable and Reduciblecited by 0
  5. 2026Locating and Steering Refusal Beyond Attentioncited by 0
  6. 2026Steering Interference Reflects the Model's Defaults, Not the Behavior Directionscited by 0
  7. 2026Navigating the digital spectrum: Assessing political bias, stability, and downstream fairness in Large Language Modelscited by 0
  8. 2026Deep Noir: Autonomous Steering Discovery via Architectural Chronometry in Transformer Modelscited by 0
  9. 2026Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisationcited by 0