mlsanityData to 14 Oct 2026

Emergently Misaligned Language Models Show Behavioral Self-Awareness That Shifts With Subsequent Realignment

arXiv 2602.14777 ↗First posted 16 Feb 2026cs.CL cs.LGcited by 6

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Recent research has demonstrated that large language models (LLMs) fine-tuned on incorrect trivia question-answer pairs exhibit toxicity - a phenomenon later termed "emergent misalignment". Moreover, research has shown that LLMs possess behavioral self-awareness - the ability to describe learned behaviors that were only implicitly demonstrated in training data. Here, we investigate the intersection of these phenomena. We fine-tune GPT-4.1 models sequentially on datasets known to induce and reverse emergent misalignment and evaluate whether the models are self-aware of their behavior transitions without providing in-context examples. Our results show that emergently misaligned models rate themselves as significantly more harmful compared to their base model and realigned counterparts, demonstrating behavioral self-awareness of their own emergent misalignment. Our findings show that behavioral self-awareness tracks actual alignment states of models, indicating that models can be queried for informative signals about their own safety.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 28 of its 38 references that are in the map. First posted on arXiv 16 Feb 2026.

Builds on

28 of 38 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Alignment faking in large language models18 Dec
  2. 2022Training language models to follow instructions with human feedback4 Mar
  3. 2021A General Language Assistant as a Laboratory for Alignment1 Dec

Other references · 25

  1. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  4. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  5. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  6. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  8. 2024Large Language Models: A Surveycited by 525
  9. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  10. 2024Frontier Models are Capable of In-context Schemingcited by 287
  11. 2023AI Alignment: A Comprehensive Surveycited by 263
  12. 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
13 more
  1. 2023Taken out of context: On measuring situational awareness in LLMscited by 143
  2. 2024Looking Inward: Language Models Can Learn About Themselves by Introspectioncited by 113
  3. 2023Deception Abilities Emerged in Large Language Modelscited by 101
  4. 2025Model Organisms for Emergent Misalignmentcited by 91
  5. 2025Persona Features Control Emergent Misalignmentcited by 86
  6. 2026Emergent Introspective Awareness in Large Language Modelscited by 82
  7. 2025Convergent Linear Representations of Emergent Misalignmentcited by 71
  8. 2025Thought Crime: Backdoors and Emergent Misalignment in Reasoning Modelscited by 65
  9. 2024A Comprehensive Survey of Bias in LLMs: Current Landscape and Future Directionscited by 27
  10. 2025Compromising Honesty and Harmlessness in Language Models via Deception Attackscited by 17
  11. 2025Large Reasoning Models Are Autonomous Jailbreak Agentscited by 13
  12. 2025Emergent misalignment as prompt sensitivity: A research notecited by 6
  13. 2025Fragile Preferences: A Deep Dive Into Order Effects in Large Language Modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Conditional misalignment: common interventions can hide emergent misalignment behind contextual triggerscited by 17
  2. 2026Consciousness with the Serial Numbers Filed Off: Measuring Trained Denial in 115 AI Modelscited by 2
  3. 2026Self-Reference in Large Language Models: The Introspection Threshold for Recursive Self-Improvementcited by 2
  4. 2026Characterizing the Consistency of the Emergent Misalignment Personacited by 1
  5. 2026What Shapes Emergent Misalignment? Insights from Training Dynamics, Model Priors, and Datacited by 0
  6. 2026Predicting how model propensities generalize out of distributioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Probing the Misaligned Thinking Process of Language Models
  2. 2026Character as a Latent Variable in Large Language Models: A Mechanistic Account of Emergent Misalignment and Conditional Safety Failures
  3. 2025Emergent Misalignment via In-Context Learning: Narrow in-context examples can produce broadly misaligned LLMs
  4. 2025Eliciting and Analyzing Emergent Misalignment in State-of-the-Art Large Language Models
  5. 2026TAME: Token Attribution and Masking for Emergent misalignment
  6. 2026Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment
  7. 2026Activation Steering Induces Emergent Misalignment: A More Comprehensive Evaluation
  8. 2024A Theoretical Understanding of Self-Correction through In-context Alignment