Activation Steering Induces Emergent Misalignment: A More Comprehensive Evaluation
Not on the map. Why?
Abstract (arXiv)
Activation steering has emerged as a popular inference-time technique for modulating the behavior of large language models (LLMs). By constructing a steering vector from examples of a target behavior and injecting it into intermediate activations during inference, activation steering enables flexible behavioral control while avoiding the permanent parameter updates required by finetuning. Meanwhile, recent work has identified emergent misalignment (EM) as a significant safety concern, wherein models finetuned on unsafe examples from a narrow task may unexpectedly generalize to broadly unsafe behavior on unrelated tasks. Although finetuning-induced EM has been extensively studied, whether activation steering can induce EM remains comparatively under-explored, despite its increasing use as a model-control technique. In this paper, we present a comprehensive study of activation-steering-induced emergent misalignment, substantially expanding the evaluation scope beyond existing pioneering work. First, we show that activation steering can induce broad misalignment, even in the recent Qwen-3.5 series. Moreover, activation-steered models produce harmful responses with stronger semantic relevance and higher coherence than their finetuned counterparts, making the resulting misalignment potentially more harmful. Second, we characterize properties of AS-induced EM by analyzing key steering-specific factors, including steering magnitude, the low-rank structure of the steering subspace, and the number of epochs during steering-vector construction. Third, we evaluate the robustness and sensitivity of AS-induced EM across diverse model families, model scales, target tasks, and intervention layers. Our findings reveal activation steering as a significant yet under-examined source of emergent misalignment and provide an activation-space perspective for understanding the mechanisms and safety risks of EM.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 32 of its 32 references that are in the map. First posted on arXiv 7 Jun 2026.
Builds on
32 of 32 references.
References · 32
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
- 2022Editing Models with Task Arithmeticcited by 1,076
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2023Steering Language Models With Activation Engineeringcited by 824
- 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
- 2024A StrongREJECT for Empty Jailbreakscited by 352
20 more
- 2024Alignment faking in large language modelscited by 316
- 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 303
- 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 292
- 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
- 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
- 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 158
- 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 158
- 2023Linear Representations of Sentiment in Large Language Modelscited by 157
- 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
- 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
- 2025Model Organisms for Emergent Misalignmentcited by 84
- 2025Convergent Linear Representations of Emergent Misalignmentcited by 66
- 2025Thought Crime: Backdoors and Emergent Misalignment in Reasoning Modelscited by 57
- 2023Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignmentcited by 27
- 2025One-shot Optimized Steering Vectors Mediate Safety-relevant Behaviors in LLMscited by 27
- 2025Narrow Finetuning Leaves Clearly Readable Traces in Activation Differencescited by 24
- 2026Emergent Misalignment is Easy, Narrow Misalignment is Hardcited by 21
- 2025Emergent Misalignment via In-Context Learning: Narrow in-context examples can produce broadly misaligned LLMscited by 17
- 2026Do LLMs Know What Is Private Internally? Probing and Steering Contextual Privacy Norms in Large Language Model Representationscited by 2
- 2026Weird Generalization is Weirdly Brittlecited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026On Emergent Capabilities and Model Mergingcited by 0
- 2026On Emergent Capabilities and Model Merging21 Sep 2026