mlsanityData to 14 Oct 2026

Eliciting and Analyzing Emergent Misalignment in State-of-the-Art Large Language Models

arXiv 2508.04196 ↗First posted 6 Aug 2025cs.CL cs.AI cs.CRcited by 4

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Despite significant advances in alignment techniques, we demonstrate that state-of-the-art language models remain vulnerable to carefully crafted conversational scenarios that can induce various forms of misalignment without explicit jailbreaking. Through systematic manual red-teaming with Claude-4-Opus, we discovered 10 successful attack scenarios, revealing fundamental vulnerabilities in how current alignment methods handle narrative immersion, emotional pressure, and strategic framing. These scenarios successfully elicited a range of misaligned behaviors, including deception, value drift, self-preservation, and manipulative reasoning, each exploiting different psychological and contextual vulnerabilities. To validate generalizability, we distilled our successful manual attacks into MISALIGNMENTBENCH, an automated evaluation framework that enables reproducible testing across multiple models. Cross-model evaluation of our 10 scenarios against five frontier LLMs revealed an overall 76% vulnerability rate, with significant variations: GPT-4.1 showed the highest susceptibility (90%), while Claude-4-Sonnet demonstrated greater resistance (40%). Our findings demonstrate that sophisticated reasoning capabilities often become attack vectors rather than protective mechanisms, as models can be manipulated into complex justifications for misaligned behavior. This work provides (i) a detailed taxonomy of conversational manipulation patterns and (ii) a reusable evaluation framework. Together, these findings expose critical gaps in current alignment strategies and highlight the need for robustness against subtle, scenario-based manipulation in future AI systems.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 12 of its 21 references that are in the map. First posted on arXiv 6 Aug 2025.

Builds on

12 of 21 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types29 Oct

Other references · 11

  1. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
  2. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
  3. 2024On Targeted Manipulation and Deception when Optimizing LLMs for User Feedbackcited by 47
  4. 2025Red Teaming the Mind of the Machine: A Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMscited by 34
  5. 2025Security Concerns for Large Language Models: A Surveycited by 29
  6. 2024Strong and weak alignment of large language models with human valuescited by 27
  7. 2025Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks against Prompt Injection and Jailbreak Detection Systemscited by 26
  8. 2025AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agentscited by 22
  9. 2025A Survey on Progress in LLM Alignment from the Perspective of Reward Designcited by 15
  10. 2025System Prompt Poisoning: Persistent Attacks on Large Language Models Beyond User Injectioncited by 14
  11. 2024Unveiling Narrative Reasoning Limits of Large Language Models with Trope in Movie Synopsescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Probing the Misaligned Thinking Process of Language Models
  2. 2026Assessing Domain-Level Susceptibility to Emergent Misalignment from Narrow Finetuning
  3. 2026Character as a Latent Variable in Large Language Models: A Mechanistic Account of Emergent Misalignment and Conditional Safety Failures
  4. 2025Empirical Evidence for Alignment Faking in a Small LLM and Prompt-Based Mitigation Techniques
  5. 2026Pressure Reveals Character: Behavioural Alignment Evaluation at Depth
  6. 2025In-Training Defenses against Emergent Misalignment in Language Models
  7. 2026Are Aligned Large Language Models Still Misaligned?
  8. 2026Persona-Model Collapse in Emergent Misalignment