# Emergently Misaligned Language Models Show Behavioral Self-Awareness That Shifts With Subsequent Realignment

arXiv: https://arxiv.org/abs/2602.14777 · HTML: https://mlsanity.com/paper/2602.14777/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 16 Feb 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)

## Builds on (most cited first, 28 of 28)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [Alignment faking in large language models](https://mlsanity.com/paper/2412.14093/) (2024)
- [Deliberative Alignment: Reasoning Enables Safer Language Models](https://mlsanity.com/paper/2412.16339/) (2024)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [AI Alignment: A Comprehensive Survey](https://mlsanity.com/paper/2310.19852/) (2023)
- [The Alignment Problem from a Deep Learning Perspective](https://mlsanity.com/paper/2209.00626/) (2022)
- [Taken out of context: On measuring situational awareness in LLMs](https://mlsanity.com/paper/2309.00667/) (2023)
- [Looking Inward: Language Models Can Learn About Themselves by Introspection](https://mlsanity.com/paper/2410.13787/) (2024)
- [Deception Abilities Emerged in Large Language Models](https://mlsanity.com/paper/2307.16513/) (2023)
- [Model Organisms for Emergent Misalignment](https://mlsanity.com/paper/2506.11613/) (2025)
- [Persona Features Control Emergent Misalignment](https://mlsanity.com/paper/2506.19823/) (2025)
- [Emergent Introspective Awareness in Large Language Models](https://mlsanity.com/paper/2601.01828/) (2026)
- [Convergent Linear Representations of Emergent Misalignment](https://mlsanity.com/paper/2506.11618/) (2025)
- [Thought Crime: Backdoors and Emergent Misalignment in Reasoning Models](https://mlsanity.com/paper/2506.13206/) (2025)
- [A Comprehensive Survey of Bias in LLMs: Current Landscape and Future Directions](https://mlsanity.com/paper/2409.16430/) (2024)
- [Compromising Honesty and Harmlessness in Language Models via Deception Attacks](https://mlsanity.com/paper/2502.08301/) (2025)
- [Large Reasoning Models Are Autonomous Jailbreak Agents](https://mlsanity.com/paper/2508.04039/) (2025)
- [Emergent misalignment as prompt sensitivity: A research note](https://mlsanity.com/paper/2507.06253/) (2025)
- [Fragile Preferences: A Deep Dive Into Order Effects in Large Language Models](https://mlsanity.com/paper/2506.14092/) (2025)

## Built on by (most cited first, 6 of 6)

- [Conditional misalignment: common interventions can hide emergent misalignment behind contextual triggers](https://mlsanity.com/paper/2604.25891/) (2026)
- [Consciousness with the Serial Numbers Filed Off: Measuring Trained Denial in 115 AI Models](https://mlsanity.com/paper/2604.25922/) (2026)
- [Self-Reference in Large Language Models: The Introspection Threshold for Recursive Self-Improvement](https://mlsanity.com/paper/2607.04277/) (2026)
- [Characterizing the Consistency of the Emergent Misalignment Persona](https://mlsanity.com/paper/2604.28082/) (2026)
- [What Shapes Emergent Misalignment? Insights from Training Dynamics, Model Priors, and Data](https://mlsanity.com/paper/2606.20814/) (2026)
- [Predicting how model propensities generalize out of distribution](https://mlsanity.com/paper/2610.14937/) (2026)

## Abstract (arXiv)

Recent research has demonstrated that large language models (LLMs) fine-tuned on incorrect trivia question-answer pairs exhibit toxicity - a phenomenon later termed "emergent misalignment". Moreover, research has shown that LLMs possess behavioral self-awareness - the ability to describe learned behaviors that were only implicitly demonstrated in training data. Here, we investigate the intersection of these phenomena. We fine-tune GPT-4.1 models sequentially on datasets known to induce and reverse emergent misalignment and evaluate whether the models are self-aware of their behavior transitions without providing in-context examples. Our results show that emergently misaligned models rate themselves as significantly more harmful compared to their base model and realigned counterparts, demonstrating behavioral self-awareness of their own emergent misalignment. Our findings show that behavioral self-awareness tracks actual alignment states of models, indicating that models can be queried for informative signals about their own safety.