mlsanityData to 7 Oct 2026

Steering Without Side Effects: Improving Post-Deployment Control of Language Models

arXiv 2406.15518 ↗First posted 21 Jun 2024cs.CL cs.LGcited by 44

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Language models (LMs) have been shown to behave unexpectedly post-deployment. For example, new jailbreaks continually arise, allowing model misuse, despite extensive red-teaming and adversarial training from developers. Given most model queries are unproblematic and frequent retraining results in unstable user experience, methods for mitigation of worst-case behavior should be targeted. One such method is classifying inputs as potentially problematic, then selectively applying steering vectors on these problematic inputs, i.e. adding particular vectors to model hidden states. However, steering vectors can also negatively affect model performance, which will be an issue on cases where the classifier was incorrect. We present KL-then-steer (KTS), a technique that decreases the side effects of steering while retaining its benefits, by first training a model to minimize Kullback-Leibler (KL) divergence between a steered and unsteered model on benign inputs, then steering the model that has undergone this training. Our best method prevents 44% of jailbreak attacks compared to the original Llama-2-chat-7B model while maintaining helpfulness (as measured by MT-Bench) on benign requests almost on par with the original LM. To demonstrate the generality and transferability of our method beyond jailbreaks, we show that our KTS model can be steered to reduce bias towards user-suggested answers on TruthfulQA. Code is available: https://github.com/AsaCooperStickland/kl-then-steer.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 25 of its 31 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

25 of 31 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Refusal in Language Models Is Mediated by a Single Direction17 Jun
  2. 2023Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignment15 Nov

Other references · 23

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  9. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  10. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  11. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  12. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
11 more
  1. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  2. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  3. 2023Steering Language Models With Activation Engineeringcited by 824
  4. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  5. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
  6. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  7. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 140
  8. 2024Coercing LLMs to do and reveal (almost) anythingcited by 88
  9. 2023Improving Activation Steering in Language Models with Mean-Centringcited by 64
  10. 2024Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thoughtcited by 37
  11. 2024Investigating Bias Representations in Llama 2 Chat via Activation Steeringcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 18; 2026: 14.

44 papers.

Sort:

By area:

  1. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  2. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
  3. 2024Sycophancy in Large Language Models: Causes and Mitigationscited by 97
  4. 2024Robust LLM safeguarding via refusal feature adversarial trainingcited by 60
  5. 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 45
  6. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  7. 2025A Unified Understanding and Evaluation of Steering Methodscited by 40
  8. 2025Understanding (Un)Reliability of Steering Vectors in Language Modelscited by 39
  9. 2025LLM Unlearning via Neural Activation Redirectioncited by 38
  10. 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safetycited by 38
  11. 2025AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defendercited by 37
  12. 2024Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Modelscited by 35
32 more
  1. 2024Evaluating the Prompt Steerability of Large Language Modelscited by 26
  2. 2024What Features in Prompts Jailbreak LLMs? Investigating the Mechanisms Behind Attackscited by 24
  3. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 23
  4. 2025The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Directioncited by 21
  5. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 18
  6. 2024Towards Inference-time Category-wise Safety Steering for Large Language Modelscited by 17
  7. 2024OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMscited by 14
  8. 2025BASIL: Bayesian Assessment of Sycophancy in LLMscited by 11
  9. 2025SafeSteer: Interpretable Safety Steering with Refusal-Evasion in LLMscited by 9
  10. 2026From Weights to Activations: Is Steering the Next Frontier of Adaptation?cited by 8
  11. 2025Disentangled Safety Adapters Enable Efficient Guardrails and Flexible Inference-Time Alignmentcited by 6
  12. 2025Hallucination reduction with CASAL: Contrastive Activation Steering For Amortized Learningcited by 6
  13. 2026Fine-Grained Activation Steering: Steering Less, Achieving Morecited by 6
  14. 2024Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviorscited by 5
  15. 2025PENDULUM: A Benchmark for Assessing Sycophancy in Multimodal Large Language Modelscited by 4
  16. 2026Sparse Autoencoders are Capable LLM Jailbreak Mitigatorscited by 4
  17. 2024Representation Tuningcited by 3
  18. 2025DSO: Direct Steering Optimization for Bias Mitigationcited by 3
  19. 2026Towards Understanding Steering Strengthcited by 3
  20. 2025Steering Language Models in Multi-Token Generation: A Case Study on Tense and Aspectcited by 2
  21. 2025EnSToM: Enhancing Dialogue Systems with Entropy-Scaled Steering Vectors for Topic Maintenancecited by 1
  22. 2025ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking Attackcited by 1
  23. 2026Knowing without Acting: The Disentangled Geometry of Safety Mechanisms in Large Language Modelscited by 1
  24. 2026Predicting Future Behaviors in Reasoning Models Enables Better Steeringcited by 1
  25. 2026Structural Instability of Feature Compositioncited by 0
  26. 2026xMIx: High-Performance Serving-Time Platform for Mechanistic Interpretability Appscited by 0
  27. 2026Safety Cost of Steering Vectors Is Separable and Reduciblecited by 0
  28. 2026Your AI, On a Dial: Controlling Investment Bias in LLMs with a Single Neuroncited by 0
  29. 2026Right Frame, Wrong Rule: Cultural Cues Expose the Financial Knowledge Gap They Were Meant to Closecited by 0
  30. 2026Write Back the Delta: Revisiting the Same Tokens with Fresh Representationscited by 0
  31. 2026When Do Model Internals Help? Exploring the Role of Representation Engineering in LLM Safetycited by 0
  32. 2026Constitutional adapters: Inference-time interventions for misalignment and misusecited by 0