mlsanityData to 7 Oct 2026

An Emergent Mirage: Is Emergent Misalignment and Realignment Indeed a Robust Phenomenon?

arXiv 2607.09053 ↗First posted 10 Jul 2026cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Recent work has reported Emergent Misalignment (EM), where language models fine-tuned on narrow, domain-specific misaligned datasets abruptly acquire broadly misaligned behavior, alongside evidence that this behavior can be reversed through limited realignment. We systematically study repeated alignment and misalignment cycles using controlled fine-tuning loops while tracking behavioral performance, and LoRA representations throughout training. Although we reproduce EM, we find that both misalignment and realignment are highly sensitive to superficial dataset characteristics, with apparent rapid realignment largely disappearing after controlling for response-length differences. We further find that previously reported mechanistic signatures, including representational phase transitions in LoRA space, do not consistently correlate with behavioral misalignment across training. Our results suggest that current evidence for EM is less robust than previously claimed and highlight the need for evaluation protocols that carefully control for these surface level dataset artifacts to identify the robustness of the EM phenomenon.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 16 of its 28 references that are in the map. First posted on arXiv 10 Jul 2026.

Builds on

16 of 28 references.

References · 16

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  4. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  5. 2023LIMA: Less Is More for Alignmentcited by 1,080
  6. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  7. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  8. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  9. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  10. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  11. 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 125
  12. 2025Convergent Linear Representations of Emergent Misalignmentcited by 66
4 more
  1. 2025School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMscited by 58
  2. 2025Thought Crime: Backdoors and Emergent Misalignment in Reasoning Modelscited by 57
  3. 2024Language Models Resist Alignment: Evidence From Data Compressioncited by 50
  4. 2025In-Training Defenses against Emergent Misalignment in Language Modelscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.