mlsanityData to 7 Oct 2026

Sugar-Coated Poison: Benign Generation Unlocks LLM Jailbreaking

arXiv 2504.05652 ↗First posted 8 Apr 2025cs.CR cs.CLcited by 5

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

With the increasingly deep integration of large language models (LLMs) across diverse domains, the effectiveness of their safety mechanisms is encountering severe challenges. Currently, jailbreak attacks based on prompt engineering have become a major safety threat. However, existing methods primarily rely on black-box manipulation of prompt templates, resulting in poor interpretability and limited generalization. To break through the bottleneck, this study first introduces the concept of Defense Threshold Decay (DTD), revealing the potential safety impact caused by LLMs' benign generation: as benign content generation in LLMs increases, the model's focus on input instructions progressively diminishes. Building on this insight, we propose the Sugar-Coated Poison (SCP) attack paradigm, which uses a "semantic reversal" strategy to craft benign inputs that are opposite in meaning to malicious intent. This strategy induces the models to generate extensive benign content, thereby enabling adversarial reasoning to bypass safety mechanisms. Experiments show that SCP outperforms existing baselines. Remarkably, it achieves an average attack success rate of 87.23% across six LLMs. For defense, we propose Part-of-Speech Defense (POSD), leveraging verb-noun dependencies for syntactic analysis to enhance safety of LLMs while preserving their generalization ability.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 27 of its 48 references that are in the map. First posted on arXiv 8 Apr 2025.

Builds on

27 of 48 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs3 Oct
  2. 2024When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search13 Jun
  3. 2024ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings25 Feb
  4. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability13 Feb
  5. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  6. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  7. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily14 Nov
  8. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  10. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  12. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 15

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2024Mixtral of Expertscited by 1,785
  5. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  6. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  7. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  8. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  9. 2021Recursively Summarizing Books with Human Feedbackcited by 287
  10. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  11. 2020Recipes for Safety in Open-domain Chatbotscited by 168
  12. 2023Aligning Large Language Models with Human Preferences through Representation Engineeringcited by 71
3 more
  1. 2023LLM-based Medical Assistant Personalization with Short- and Long-Term Memory Coordinationcited by 46
  2. 2024PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairscited by 11
  3. 2025Parameter-Efficient Fine-Tuning of Large Language Models via Deconvolution in Subspacecited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.