Sugar-Coated Poison: Benign Generation Unlocks LLM Jailbreaking
On the map: LLM safety and security → Jailbreaking
Abstract (arXiv)
With the increasingly deep integration of large language models (LLMs) across diverse domains, the effectiveness of their safety mechanisms is encountering severe challenges. Currently, jailbreak attacks based on prompt engineering have become a major safety threat. However, existing methods primarily rely on black-box manipulation of prompt templates, resulting in poor interpretability and limited generalization. To break through the bottleneck, this study first introduces the concept of Defense Threshold Decay (DTD), revealing the potential safety impact caused by LLMs' benign generation: as benign content generation in LLMs increases, the model's focus on input instructions progressively diminishes. Building on this insight, we propose the Sugar-Coated Poison (SCP) attack paradigm, which uses a "semantic reversal" strategy to craft benign inputs that are opposite in meaning to malicious intent. This strategy induces the models to generate extensive benign content, thereby enabling adversarial reasoning to bypass safety mechanisms. Experiments show that SCP outperforms existing baselines. Remarkably, it achieves an average attack success rate of 87.23% across six LLMs. For defense, we propose Part-of-Speech Defense (POSD), leveraging verb-noun dependencies for syntactic analysis to enhance safety of LLMs while preserving their generalization ability.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 27 of its 48 references that are in the map. First posted on arXiv 8 Apr 2025.
Builds on
27 of 48 references.
Closest ancestors · 12
Same direction, newest first
- 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs3 Oct
- 2024When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search13 Jun
- 2024ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings25 Feb
- 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability13 Feb
- 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
- 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
- 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily14 Nov
- 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
- 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
- 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
- 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
Other references · 15
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2024Mixtral of Expertscited by 1,785
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
- 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
- 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
- 2021Recursively Summarizing Books with Human Feedbackcited by 287
- 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
- 2020Recipes for Safety in Open-domain Chatbotscited by 168
- 2023Aligning Large Language Models with Human Preferences through Representation Engineeringcited by 71
3 more
- 2023LLM-based Medical Assistant Personalization with Short- and Long-Term Memory Coordinationcited by 46
- 2024PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairscited by 11
- 2025Parameter-Efficient Fine-Tuning of Large Language Models via Deconvolution in Subspacecited by 11
Built on by
Citing papers per year: 2025: 1; 2026: 4.
5 papers.
- 2026MI-PRUN: Optimize Large Language Model Pruning via Mutual Informationcited by 2
- 2026Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarkingcited by 2
- 2026The Struggle Between Continuation and Refusal: A Mechanistic Analysis of the Continuation-Triggered Jailbreak in LLMscited by 1
- 2025BreakFun: Jailbreaking LLMs via Object Instantiation under Simulated Code Executioncited by 0
- 2026Model-Agnostic Lifelong LLM Safety via Externalized Attack-Defense Co-Evolutioncited by 0
- 2026Model-Agnostic Lifelong LLM Safety via Externalized Attack-Defense Co-Evolution13 May 2026
- 2026The Struggle Between Continuation and Refusal: A Mechanistic Analysis of the Continuation-Triggered Jailbreak in LLMs9 Mar 2026
- 2026Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarking27 Feb 2026
- 2026MI-PRUN: Optimize Large Language Model Pruning via Mutual Information12 Jan 2026
- 2025BreakFun: Jailbreaking LLMs via Object Instantiation under Simulated Code Execution19 Oct 2025