Data to 7 Oct 2026

FuzzLLM: A Novel and Universal Fuzzing Framework for Proactively Discovering Jailbreak Vulnerabilities in Large Language Models

arXiv 2309.05274 ↗First posted 11 Sep 2023cs.CRcited by 46

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Jailbreak vulnerabilities in Large Language Models (LLMs), which exploit meticulously crafted prompts to elicit content that violates service guidelines, have captured the attention of research communities. While model owners can defend against individual jailbreak prompts through safety training strategies, this relatively passive approach struggles to handle the broader category of similar jailbreaks. To tackle this issue, we introduce FuzzLLM, an automated fuzzing framework designed to proactively test and discover jailbreak vulnerabilities in LLMs. We utilize templates to capture the structural integrity of a prompt and isolate key features of a jailbreak class as constraints. By integrating different base classes into powerful combo attacks and varying the elements of constraints and prohibited questions, FuzzLLM enables efficient testing with reduced manual effort. Extensive experiments demonstrate FuzzLLM's effectiveness and comprehensiveness in vulnerability discovery across various LLMs.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. First posted on arXiv 11 Sep 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 17; 2025: 19; 2026: 6.

46 papers.

Sort:

By area:

2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
2023Can LLM-Generated Misinformation Be Detected?cited by 173
2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
2024WIPI: A New Web Threat for LLM-Driven Web Agentscited by 50
2024Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledgecited by 48

34 more

2023Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attackscited by 30
2024PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMscited by 20
2024You Know What I'm Saying: Jailbreak Attack via Implicit Referencecited by 13
2024PathSeeker: Exploring LLM Security Vulnerabilities with a Reinforcement Learning-Based Jailbreak Approachcited by 9
2025GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methodscited by 9
2025PromptPex: Automatic Test Generation for Language Model Promptscited by 9
2024Can LLMs Deeply Detect Complex Malicious Queries? A Framework for Jailbreaking via Obfuscating Intentcited by 8
2024Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attackscited by 8
2024AdaPPA: Adaptive Position Pre-Fill Jailbreak Attack Approach Targeting LLMscited by 8
2024Recent advancements in LLM Red-Teaming: Techniques, Defenses, and Ethical Considerationscited by 8
2025Evolving Security in LLMs: A Study of Jailbreak Attacks and Defensescited by 8
2025EVOREFUSE: Evolutionary Prompt Optimization for Evaluation and Mitigation of LLM Over-Refusal to Pseudo-Malicious Instructionscited by 6
2025ASTRA: Autonomous Spatial-Temporal Red-teaming for AI Software Assistantscited by 6
2025ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusalcited by 6
2025Bypassing Prompt Guards in Production with Controlled-Release Promptingcited by 6
2026Knowledge-Driven Multi-Turn Jailbreaking on Large Language Modelscited by 6
2024Towards Action Hijacking of Large Language Model-based Agentcited by 4
2025SafeLLM: Unlearning Harmful Outputs from Large Language Models against Jailbreak Attackscited by 4
2025SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defensescited by 4
2026Enhancing Instruction Following of LLMs via Activation Steering with Dynamic Rejectioncited by 4
2025Breaking the Loop: Detecting and Mitigating Denial-of-Service Vulnerabilities in Large Language Modelscited by 3
2025StructTransform: A Scalable Attack Surface for Safety-Aligned Large Language Modelscited by 2
2025Jailbreaking LLMs via Semantically Relevant Nested Scenarios with Targeted Toxic Knowledgecited by 2
2025Jailbreaking in the Haystackcited by 2
2025Strategic Deflection: Defending LLMs from Logit Manipulationcited by 1
2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
2025HFuzzer: Testing Large Language Models for Package Hallucinations via Phrase-based Fuzzingcited by 1
2025MSCR: Exploring the Vulnerability of LLMs' Mathematical Reasoning Abilities Using Multi-Source Candidate Replacementcited by 1
2026Not All Tokens Are Created Equal: Query-Efficient Jailbreak Fuzzing for LLMscited by 1
2026One Word at a Time: Incremental Completion Decomposition Breaks LLM Safetycited by 1
2026EvoJail: Evolutionary Diverse Jailbreak Prompt Generation for Large Language Modelscited by 1
2025TestDG: Test-time Domain Generalization for Continual Test-time Adaptationcited by 0
2026Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Modelscited by 0