Misalignment Has a Personality: A Big Five Account of Emergent Misalignment
Not on the map. Why?
Abstract (arXiv)
Fine-tuning a language model on data containing a narrow flaw, such as insecure code or incorrect mathematical answers, can cause broad misalignment through a mechanism that remains debated. We provide an interpretable account: in the models and corpora we study, misalignment behaves like a shift in personality. Prior work extracts activation directions for character traits from a single binary contrast, which can separate or steer behavior without establishing a calibrated scale. We instead extract personality vectors for the Big Five using a graded, three-level intervention and validate them on two open-weight models. The three levels are linearly ordered, with Cohen's d values of up to 6.2; the vectors transfer zero-shot and trait-specifically to an independent corpus; and their effects are strongest within a middle-layer band. Applied to training data, the vectors reveal that misaligned corpora across eight domains share a common Big Five signature: lower agreeableness and conscientiousness, together with higher extraversion and neuroticism. This signature is recovered by both models with a correlation of r = 0.94. Fine-tuning imprints the same profile, shifting the model's generations along the corresponding signature, with r = 0.83 using activation-based measurements and r = 0.90 using a text-based judge, while also shifting internal activations with r = 0.69. The same vectors characterize sycophancy as high extraversion and low conscientiousness rather than excess agreeableness, a distinction that a single direction cannot capture. Calibrated personality vectors transform an opaque safety phenomenon into a human-legible diagnostic profile.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 35 references that are in the map. First posted on arXiv 29 Jul 2026.
Builds on
18 of 35 references.
References · 18
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
- 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 158
6 more
- 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 114
- 2024BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Datacited by 49
- 2026PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebracited by 11
- 2025Vibe Check: Understanding the Effects of LLM-Based Conversational Agents' Personality and Alignment on User Perceptions in Goal-Oriented Taskscited by 4
- 2026Stories of Your Life as Others: A Round-Trip Evaluation of LLM-Generated Life Stories Conditioned on Rich Psychometric Profilescited by 2
- 2026Too Nice to Tell the Truth: Quantifying Agreeableness-Driven Sycophancy in Role-Playing Language Modelscited by 2
Built on by
Not cited yet.