mlsanityData to 14 Oct 2026

GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMs

arXiv 2411.14133 ↗First posted 21 Nov 2024cs.LG cs.AI cs.CRcited by 12

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

LLMs have shown impressive capabilities across various natural language processing tasks, yet remain vulnerable to input prompts, known as jailbreak attacks, carefully designed to bypass safety guardrails and elicit harmful responses. Traditional methods rely on manual heuristics but suffer from limited generalizability. Despite being automatic, optimization-based attacks often produce unnatural prompts that can be easily detected by safety filters or require high computational costs due to discrete token optimization. In this paper, we introduce Generative Adversarial Suffix Prompter (GASP), a novel automated framework that can efficiently generate human-readable jailbreak prompts in a fully black-box setting. In particular, GASP leverages latent Bayesian optimization to craft adversarial suffixes by efficiently exploring continuous latent embedding spaces, gradually optimizing the suffix prompter to improve attack efficacy while balancing prompt coherence via a targeted iterative refinement procedure. Through comprehensive experiments, we show that GASP can produce natural adversarial prompts, significantly improving jailbreak success over baselines, reducing training times, and accelerating inference speed, thus making it an efficient and scalable solution for red-teaming LLMs.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 61 of its 80 references that are in the map. First posted on arXiv 21 Nov 2024.

Builds on

61 of 80 references.

Closest ancestors · 25

Same direction, newest first

  1. 2024Faster-GCG: Efficient Discrete Optimization Jailbreak Attacks against Aligned Large Language Models20 Oct
  2. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs3 Oct
  3. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs21 Apr
  4. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs11 Apr
  5. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks2 Mar
  6. 2024Accelerating Greedy Coordinate Gradient and General Prompt Optimization via Probe Sampling2 Mar
  7. 2024A StrongREJECT for Empty Jailbreaks15 Feb
  8. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding14 Feb
  9. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability13 Feb
  10. 2024GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Models5 Feb
  11. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  12. 2023Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information20 Nov
13 more
  1. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization15 Nov
  2. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models23 Oct
  3. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks16 Oct
  4. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  5. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  6. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  7. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  8. 2023Low-Resource Languages Jailbreak GPT-43 Oct
  9. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  11. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots16 Jul
  12. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
  13. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study23 May

Other references · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  10. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  11. 2023A Survey of Large Language Modelscited by 2,576
  12. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
24 more
  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  2. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  3. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  4. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  5. 2018A Tutorial on Bayesian Optimizationcited by 744
  6. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  7. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  8. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  9. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 429
  10. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
  11. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 191
  12. 2023Can LLM-Generated Misinformation Be Detected?cited by 177
  13. 2023Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigationscited by 158
  14. 2023Black-Box Prompt Optimization: Aligning Large Language Models without Model Trainingcited by 122
  15. 2023An LLM can Fool Itself: A Prompt-Based Adversarial Attackcited by 116
  16. 2023Unveiling the Implicit Toxicity in Large Language Modelscited by 51
  17. 2024Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Modelscited by 46
  18. 2021Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spacescited by 45
  19. 2024How (un)ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queriescited by 27
  20. 2023Robust Safety Classifier for Large Language Models: Adversarial Prompt Shieldcited by 25
  21. 2023FFT: Towards Harmlessness Evaluation and Analysis for LLMs with Factuality, Fairness, Toxicitycited by 25
  22. 2023Advancing Bayesian Optimization via Learning Correlated Latent Spacecited by 18
  23. 2022Optimizing Training Trajectories in Variational Autoencoders via Latent Bayesian Optimization Approachcited by 7
  24. 2023Prompts have evil twinscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 2.

12 papers.

  1. 2025Understanding Refusal in Language Models with Sparse Autoencoderscited by 29
  2. 2025Logic Jailbreak: Efficiently Unlocking LLM Safety Restrictions Through Formal Logical Expressioncited by 16
  3. 2025Towards the Worst-case Robustness of Large Language Modelscited by 7
  4. 2025Mitigating Jailbreaks with Intent-Aware LLMscited by 4
  5. 2025OffTopicEval: When Large Language Models Enter the Wrong Chat, Almost Always!cited by 4
  6. 2025ALMGuard: Safety Shortcuts and Where to Find Them as Guardrails for Audio-Language Modelscited by 4
  7. 2025Efficient and Stealthy Jailbreak Attacks via Adversarial Prompt Distillation from LLMs to SLMscited by 3
  8. 2025Jailbreaking with Universal Multi-Promptscited by 2
  9. 2026ExpGuard: LLM Content Moderation in Specialized Domainscited by 2
  10. 2025PARASITE: Conditional System Prompt Poisoning to Hijack LLMscited by 0
  11. 2025Distillability of LLM Security Logic: Predicting Attack Success Rate of Outline Filling Attack via Ranking Regressioncited by 0
  12. 2026LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Deciphering the Chaos: Enhancing Jailbreak Attacks via Adversarial Prompt Translation
  2. 2026Adaptive Prompt Embedding Optimization for LLM Jailbreaking
  3. 2025LatentBreak: Jailbreaking Large Language Models through Latent Space Feedback
  4. 2024An Optimizable Suffix Is Worth A Thousand Templates: Efficient Black-box Jailbreaking without Affirmative Phrases via LLM as Optimizer
  5. 2024Adversarial Tuning: Defending Against Jailbreak Attacks for LLMs
  6. 2025LARGO: Latent Adversarial Reflection through Gradient Optimization for Jailbreaking LLMs
  7. 2025Anyone Can Jailbreak: Prompt-Based Attacks on LLMs and T2Is
  8. 2026The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoring