Transplanting, inverting, and preventing a misalignment persona: method-conditional emergent misalignment in Qwen2.5
Not on the map. Why?
Abstract (arXiv)
Emergent misalignment (EM) --- the broad misbehaviour a language model acquires after fine-tuning on narrow harmful data --- is mediated in Qwen2.5 models by a latent persona direction, and that direction is causal in open weights. Transplanting it into a model that shares only pretraining with its source induces broad EM (2.83 ± 0.26% misaligned against a random-direction floor of ~1.1%), and ablating a model's own direction roughly halves an overt inducer's broadcast (21% to 10%). The transplant doubles as a measurement method, causally assaying directions that a source model represents but cannot itself express. Whether a fine-tune recruits this persona depends on method and capacity, and since low-rank PEFT is the cheaper regime at scale, the recruiting method is also the economical one. On Qwen2.5-32B, LoRA at low ranks on insecure code recruits it (3.4% misaligned) while full SFT on identical data does not (0.3%) and moves against the persona axis (drift--persona cosine +0.17 at rank 1 to -0.10), the far-inducer, high-capacity exception consistent with a representational-distance × capacity account. The persona's causal role is itself conditional. Steering a bad-medical SFT run away from the direction during training raises the broadcast from ~24% to ~50% while matched random controls stay at or below baseline, replicated across three training seeds, so removing the direction is no blanket recipe. Because recruitment is a loss-reducing shortcut that capacity renders redundant, it can be screened for and prevented in the tested instances. Persona loss-relevance at the SFT solution orders four inducers' broadcasts rank-perfectly within Qwen2.5, inoculation removes recruitment selectively (4.75% to 0.0%, code coherence 65% to 87%), and fine-tuning orthogonal to the single behaviour-derived axis reduces it persona-specifically.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 18 references that are in the map. First posted on arXiv 5 Jul 2026.
Builds on
16 of 18 references.
References · 16
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
- 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
- 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
- 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
- 2024LoRA Learns Less and Forgets Lesscited by 313
- 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 303
- 2023A Rank Stabilization Scaling Factor for Fine-Tuning with LoRAcited by 231
- 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 158
- 2024LoRA vs Full Fine-tuning: An Illusion of Equivalencecited by 102
- 2025Convergent Linear Representations of Emergent Misalignmentcited by 66
- 2025Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuningcited by 34
- 2025Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-timecited by 32
4 more
- 2026Emergent Misalignment is Easy, Narrow Misalignment is Hardcited by 21
- 2025Emergent Misalignment via In-Context Learning: Narrow in-context examples can produce broadly misaligned LLMscited by 17
- 2025Re-Emergent Misalignment: How Narrow Fine-Tuning Erodes Safety Alignment in LLMscited by 9
- 2026Understanding Emergent Misalignment via Feature Superposition Geometrycited by 6
Built on by
Not cited yet.