mlsanityData to 7 Oct 2026

Graph of Attacks with Pruning: Optimizing Stealthy Jailbreak Prompt Generation for Enhanced LLM Content Moderation

arXiv 2501.18638 ↗First posted 28 Jan 2025cs.CR cs.AI cs.CLcited by 9

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

As large language models (LLMs) become increasingly prevalent, ensuring their robustness against adversarial misuse is crucial. This paper introduces the GAP (Graph of Attacks with Pruning) framework, an advanced approach for generating stealthy jailbreak prompts to evaluate and enhance LLM safeguards. GAP addresses limitations in existing tree-based LLM jailbreak methods by implementing an interconnected graph structure that enables knowledge sharing across attack paths. Our experimental evaluation demonstrates GAP's superiority over existing techniques, achieving a 20.8% increase in attack success rates while reducing query costs by 62.7%. GAP consistently outperforms state-of-the-art methods for attacking both open and closed LLMs, with attack success rates of >96%. Additionally, we present specialized variants like GAP-Auto for automated seed generation and GAP-VLM for multimodal attacks. GAP-generated prompts prove highly effective in improving content moderation systems, increasing true positive detection rates by 108.5% and accuracy by 183.6% when used for fine-tuning. Our implementation is available at https://github.com/dsbuddy/GAP-LLM-Safety.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 24 of its 40 references that are in the map. First posted on arXiv 28 Jan 2025.

Builds on

24 of 40 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs3 Oct
  2. 2024FlipAttack: Jailbreak LLMs via Flipping2 Oct
  3. 2024PathSeeker: Exploring LLM Security Vulnerabilities with a Reinforcement Learning-Based Jailbreak Approach21 Sep
  4. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models28 Mar
  5. 2024PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Rails24 Feb
  6. 2024Attacking Large Language Models with Projected Gradient Descent14 Feb
  7. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability13 Feb
  8. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  9. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily14 Nov
  10. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  11. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  12. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
4 more
  1. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  2. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models7 Aug
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  4. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 8

  1. 2022Red Teaming Language Models with Language Modelscited by 983
  2. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
  3. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  4. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 224
  5. 2024Curiosity-driven Red-teaming for Large Language Modelscited by 83
  6. 2023BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPTcited by 70
  7. 2024RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agentcited by 39
  8. 2023Exploring Vision-Language Models for Imbalanced Learningcited by 35

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 4.

9 papers.

  1. 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflowscited by 57
  2. 2025SoK: Evaluating Jailbreak Guardrails for Large Language Modelscited by 17
  3. 2025Red-Bandit: Test-Time Adaptation for LLM Red-Teaming via Bandit-Guided LoRA Expertscited by 3
  4. 2026Chunky Post-Training: Data Driven Failures of Generalizationcited by 3
  5. 2025Graph of Attacks: Improved Black-Box and Interpretable Jailbreaks for LLMscited by 2
  6. 2025Efficient and Stealthy Jailbreak Attacks via Adversarial Prompt Distillation from LLMs to SLMscited by 2
  7. 2026Redefining AI Red Teaming in the Agentic Era: From Weeks to Hourscited by 1
  8. 2026EvoHarmBench: Breaking Content Moderation with Iterative Human-Like Evasioncited by 0
  9. 2026CodeMimicry: Exploiting Safety Generalization Lag in Large Language Models via Structured Code Completioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Pruning for Protection: Increasing Jailbreak Resistance in Aligned LLMs Without Fine-Tuning
  2. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey
  3. 2026The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoring
  4. 2025Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers
  5. 2026MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety
  6. 2025Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval
  7. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner
  8. 2025Jailbreaking Attacks vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race?