FreakOut-LLM: The Effect of Emotional Stimuli on Safety Alignment
On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks
also in 1 more
Abstract (arXiv)
Safety-aligned LLMs go through refusal training to reject harmful requests, but whether these mechanisms remain effective under emotionally charged stimuli is unexplored. We introduce FreakOut-LLM, a framework investigating whether emotional context compromises safety alignment in adversarial settings. Using validated psychological stimuli, we evaluate how emotional priming through system prompts affects jailbreak susceptibility across ten LLMs. We test three conditions (stress, relaxation, neutral) using scenarios from established psychological protocols, plus a no-prompt baseline, and evaluate attack success using HarmBench on AdvBench prompts. Stress priming increases jailbreak success by 65.2% compared to neutral conditions (z = 5.93, p < 0.001; OR = 1.67, Cohen's d = 0.28), while relaxation priming produces no effect (p = 0.84). Five of ten models show significant vulnerability, with the largest effects concentrated in open-weight models. Logistic regression on 59,800 queries confirms stress as the sole significant condition predictor after controlling for prompt length (p = 0.61) and model identity. Measured psychological state strongly predicts attack success (|r|≥0.70 across five instruments; all p < 0.001 in individual-level logistic regression). These results establish emotional context as a measurable attack surface with implications for real-world AI deployment in high-stress domains.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 14 of its 28 references that are in the map. First posted on arXiv 5 Apr 2026.
Builds on
14 of 28 references.
Closest ancestors · 1
Same direction, newest first
Other references · 13
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
- 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
- 2023Large Language Models Understand and Can be Enhanced by Emotional Stimulicited by 143
- 2025Agentic Misalignment: How LLMs Could Be Insider Threatscited by 136
- 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
- 2024RigorLLM: Resilient Guardrails for Large Language Models against Undesired Contentcited by 65
- 2024NegativePrompt: Leveraging Psychology for Large Language Models Enhancement via Negative Emotional Stimulicited by 18
- 2023Inducing anxiety in large language models can induce biascited by 10
- 2024Assessment and manipulation of latent constructs in pre-trained language models using psychometric scalescited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Understanding the Effects of Safety Unalignment on Large Language Models
- 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States
- 2025Scaling Patterns in Adversarial Alignment: Evidence from Multi-LLM Jailbreak Experiments
- 2026Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models
- 2025The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions
- 2026Efficient Safety Alignment of Language Models via Latent Personality Traits
- 2026EmoRSS: Mitigating Emotion-Induced Over-Refusal in Large Language Models
- 2025The Emotional Baby Is Truly Deadly: Does your Multimodal Large Reasoning Model Have Emotional Flattery towards Humans?