# Refusal is Not Safety! Benchmarking Latent Safety Risks of LLM-Driven Content Humorization

arXiv: https://arxiv.org/abs/2607.15977 · HTML: https://mlsanity.com/paper/2607.15977/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security.
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 17 Jul 2026.

## Placed in

- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 14 of 14)

- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents](https://mlsanity.com/paper/2410.09024/) (2024)
- [Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge](https://mlsanity.com/paper/2410.02736/) (2024)
- ["Humans welcome to observe": A First Look at the Agent Social Network Moltbook](https://mlsanity.com/paper/2602.10127/) (2026)
- [Adjacent Words, Divergent Intents: Jailbreaking Large Language Models via Task Concurrency](https://mlsanity.com/paper/2510.21189/) (2025)
- [Innovative Thinking, Infinite Humor: Humor Research of Large Language Models through Structured Thought Leaps](https://mlsanity.com/paper/2410.10370/) (2024)
- [Malicious LLM-Based Conversational AI Makes Users Reveal Personal Information](https://mlsanity.com/paper/2506.11680/) (2025)
- [TRACE: Task-Aware Adaptive Self-Evolving Agentic Jailbreaking](https://mlsanity.com/paper/2605.30883/) (2026)
- [HumorReject: Decoupling LLM Safety from Refusal Prefix via A Little Humor](https://mlsanity.com/paper/2501.13677/) (2025)
- [Pun Unintended: LLMs and the Illusion of Humor Understanding](https://mlsanity.com/paper/2509.12158/) (2025)
- [RedVisor: Reasoning-Aware Prompt Injection Defense via Zero-Copy KV Cache Reuse](https://mlsanity.com/paper/2602.01795/) (2026)
- [Harm or Humor: A Multimodal, Multilingual Benchmark for Overt and Covert Harmful Humor](https://mlsanity.com/paper/2603.17759/) (2026)
- [Agent Data Injection Attacks are Realistic Threats to AI Agents](https://mlsanity.com/paper/2607.05120/) (2026)

## Abstract (arXiv)

Safety defenses for large language models (LLMs) have been extensively studied, with existing approaches focusing on attack detection and refusal mechanisms. Such fixed-form direct refusal strategies may introduce the risk of prefix injection attacks. Recent work has explored a new direction that leverages humor as an indirect refusal mechanism to mitigate over-refusal in jailbreak scenarios and reduce prefix injection risks. However, this approach implicitly assumes that humorous responses are safe. Whether humorization itself introduces safety risks remains unexplored. To address this issue, we conduct an exploratory study involving over 30,000 real-world agent interaction records and 45 stand-up comedians, revealing practical safety concerns in LLM-based content humorization. Motivated by these findings, we propose textscHumorSafe, a novel framework for evaluating latent safety risk propagation during humorization. textscHumorSafe enables LLMs to learn harmful humorization patterns and use them to transform benign content into humorous content with safety risks. Across five frontier LLMs, we find that LLMs can introduce stereotypes and toxicity during humorization. We further propose textscHumorPIA, a prompt injection attack that exploits latent risks in humor-based defenses. textscHumorPIA preserves the appearance of safe humorous refusal while covertly injecting harmful content, allowing latent risks to evade existing detection mechanisms. Experiments show that it increases toxicity by 3.14× while maintaining an apparent safety rate of 97.8% even under defense settings. Our findings highlight a gap in existing LLM safety evaluations under humorized settings.