mlsanityData to 14 Oct 2026

Steering Vectors are an Adversarial Attack Surface

arXiv 2606.05958 ↗First posted 4 Jun 2026cs.LGcited by 0

On the map: LLM safety and security → Safety alignment, refusal and guardrails

also in 1 more
Abstract (arXiv)

Activation steering has become a popular way to control Large Language Model (LLM) behavior without fine-tuning. Since the technique is plug-and-play, users share datasets and precomputed vectors to steer model activations. However, we show that a stealth data poisoning attack silently compromises this pipeline. By substituting 4-6% of tokens in the steering dataset, an attacker can silently align the resulting vector with an anti-refusal direction. This jailbreaks the target model while preserving the intended steering effect on benign prompts. Under this threat model, a malicious actor can distribute an apparently safe bundle containing texts, vectors, and weights, alongside an equivalence certificate that the end-user can verify. We test the attack on two open-weight model families and eight model-attribute combinations, observing that poisoned vectors reach an absolute attack success rate (ASR) of 20-55%, +19% to +51% over a clean reference. Finally, we find that a refusal-direction orthogonalization defense can recover ≈82% of the ASR gap without harming benign behavior.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 13 of its 25 references that are in the map. First posted on arXiv 4 Jun 2026.

Builds on

13 of 25 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Refusal in Language Models Is Mediated by a Single Direction17 Jun

Other references · 12

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  3. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  4. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  6. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  7. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  8. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  9. 2023Steering Language Models With Activation Engineeringcited by 922
  10. 2016Bag of Tricks for Efficient Text Classificationcited by 876
  11. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
  12. 2023Poisoning Language Models During Instruction Tuningcited by 269

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025The Rogue Scalpel: Activation Steering Compromises LLM Safety
  2. 2026Analysing the Safety Pitfalls of Steering Vectors
  3. 2026Adversarial Robustness of Activation Steering in Large Language Models
  4. 2026Safety Cost of Steering Vectors Is Separable and Reducible
  5. 2026Steering Externalities: Benign Activation Steering Unintentionally Increases Jailbreak Risk for Large Language Models
  6. 2025AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender
  7. 2024Steering Without Side Effects: Improving Post-Deployment Control of Language Models
  8. 2026Selective Steering: Norm-Preserving Control Through Discriminative Layer Selection