# Graph of Attacks with Pruning: Optimizing Stealthy Jailbreak Prompt Generation for Enhanced LLM Content Moderation

arXiv: https://arxiv.org/abs/2501.18638 · HTML: https://mlsanity.com/paper/2501.18638/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 28 Jan 2025.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 24 of 24)

- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Tree of Attacks: Jailbreaking Black-Box LLMs Automatically](https://mlsanity.com/paper/2312.02119/) (2023)
- [GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts](https://mlsanity.com/paper/2309.10253/) (2023)
- [JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models](https://mlsanity.com/paper/2404.01318/) (2024)
- ["Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models](https://mlsanity.com/paper/2308.03825/) (2023)
- [GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher](https://mlsanity.com/paper/2308.06463/) (2023)
- [FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts](https://mlsanity.com/paper/2311.05608/) (2023)
- [DeepInception: Hypnotize Large Language Model to Be Jailbreaker](https://mlsanity.com/paper/2311.03191/) (2023)
- [A Holistic Approach to Undesired Content Detection in the Real World](https://mlsanity.com/paper/2208.03274/) (2022)
- [A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily](https://mlsanity.com/paper/2311.08268/) (2023)
- [ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation](https://mlsanity.com/paper/2310.17389/) (2023)
- [AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs](https://mlsanity.com/paper/2410.05295/) (2024)
- [COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability](https://mlsanity.com/paper/2402.08679/) (2024)
- [Attacking Large Language Models with Projected Gradient Descent](https://mlsanity.com/paper/2402.09154/) (2024)
- [Curiosity-driven Red-teaming for Large Language Models](https://mlsanity.com/paper/2402.19464/) (2024)
- [FlipAttack: Jailbreak LLMs via Flipping](https://mlsanity.com/paper/2410.02832/) (2024)
- [BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT](https://mlsanity.com/paper/2304.12298/) (2023)
- [PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Rails](https://mlsanity.com/paper/2402.15911/) (2024)
- [RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agent](https://mlsanity.com/paper/2407.16667/) (2024)
- [Exploring Vision-Language Models for Imbalanced Learning](https://mlsanity.com/paper/2304.01457/) (2023)
- [PathSeeker: Exploring LLM Security Vulnerabilities with a Reinforcement Learning-Based Jailbreak Approach](https://mlsanity.com/paper/2409.14177/) (2024)

## Built on by (most cited first, 11 of 11)

- [From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows](https://mlsanity.com/paper/2506.23260/) (2025)
- [SoK: Evaluating Jailbreak Guardrails for Large Language Models](https://mlsanity.com/paper/2506.10597/) (2025)
- [Chunky Post-Training: Data Driven Failures of Generalization](https://mlsanity.com/paper/2602.05910/) (2026)
- [Red-Bandit: Test-Time Adaptation for LLM Red-Teaming via Bandit-Guided LoRA Experts](https://mlsanity.com/paper/2510.07239/) (2025)
- [Efficient and Stealthy Jailbreak Attacks via Adversarial Prompt Distillation from LLMs to SLMs](https://mlsanity.com/paper/2506.17231/) (2025)
- [Evolving Skill-Structured Attack Memory Enhances LLM Jailbreaking](https://mlsanity.com/paper/2605.29237/) (2026)
- [Graph of Attacks: Improved Black-Box and Interpretable Jailbreaks for LLMs](https://mlsanity.com/paper/2504.19019/) (2025)
- [Redefining AI Red Teaming in the Agentic Era: From Weeks to Hours](https://mlsanity.com/paper/2605.04019/) (2026)
- [TrapSuffix: Proactive Defense Against Adversarial Suffixes in Jailbreaking](https://mlsanity.com/paper/2602.06630/) (2026)
- [EvoHarmBench: Breaking Content Moderation with Iterative Human-Like Evasion](https://mlsanity.com/paper/2608.27844/) (2026)
- [CodeMimicry: Exploiting Safety Generalization Lag in Large Language Models via Structured Code Completion](https://mlsanity.com/paper/2609.39902/) (2026)

## Abstract (arXiv)

As large language models (LLMs) become increasingly prevalent, ensuring their robustness against adversarial misuse is crucial. This paper introduces the GAP (Graph of Attacks with Pruning) framework, an advanced approach for generating stealthy jailbreak prompts to evaluate and enhance LLM safeguards. GAP addresses limitations in existing tree-based LLM jailbreak methods by implementing an interconnected graph structure that enables knowledge sharing across attack paths. Our experimental evaluation demonstrates GAP's superiority over existing techniques, achieving a 20.8% increase in attack success rates while reducing query costs by 62.7%. GAP consistently outperforms state-of-the-art methods for attacking both open and closed LLMs, with attack success rates of >96%. Additionally, we present specialized variants like GAP-Auto for automated seed generation and GAP-VLM for multimodal attacks. GAP-generated prompts prove highly effective in improving content moderation systems, increasing true positive detection rates by 108.5% and accuracy by 183.6% when used for fine-tuning. Our implementation is available at https://github.com/dsbuddy/GAP-LLM-Safety.