mlsanityData to 7 Oct 2026

Fast Adversarial Attacks on Language Models In One GPU Minute

arXiv 2402.15570 ↗First posted 23 Feb 2024cs.CR cs.AI cs.CLcited by 85

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

In this paper, we introduce a novel class of fast, beam search-based adversarial attack (BEAST) for Language Models (LMs). BEAST employs interpretable parameters, enabling attackers to balance between attack speed, success rate, and the readability of adversarial prompts. The computational efficiency of BEAST facilitates us to investigate its applications on LMs for jailbreaking, eliciting hallucinations, and privacy attacks. Our gradient-free targeted attack can jailbreak aligned LMs with high attack success rates within one minute. For instance, BEAST can jailbreak Vicuna-7B-v1.5 under one minute with a success rate of 89% when compared to a gradient-based baseline that takes over an hour to achieve 70% success rate using a single Nvidia RTX A6000 48GB GPU. Additionally, we discover a unique outcome wherein our untargeted attack induces hallucinations in LM chatbots. Through human evaluations, we find that our untargeted attack causes Vicuna-7B-v1.5 to produce ~15% more incorrect outputs when compared to LM outputs in the absence of our attack. We also learn that 22% of the time, BEAST causes Vicuna to generate outputs that are not relevant to the original prompt. Further, we use BEAST to generate adversarial prompts in a few seconds that can boost the performance of existing membership inference attacks for LMs. We believe that our fast attack, BEAST, has the potential to accelerate research in LM security and privacy. Our codebase is publicly available at https://github.com/vinusankars/BEAST.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 56 of its 64 references that are in the map. First posted on arXiv 23 Feb 2024.

Builds on

56 of 64 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  2. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
  3. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  4. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  5. 2023Certifying LLM Safety against Adversarial Prompting6 Sep
  6. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Models4 Sep
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  8. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 48

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2017Deep reinforcement learning from human preferencescited by 3,846
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  11. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  12. 2023Mistral 7Bcited by 3,065
36 more
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2020Extracting Training Data from Large Language Modelscited by 2,144
  4. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  5. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  7. 2021Membership Inference Attacks From First Principlescited by 839
  8. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  9. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  10. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  11. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  12. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  13. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  14. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  15. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  16. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  17. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  18. 2023The Falcon Series of Open Language Modelscited by 526
  19. 2018Generating Natural Language Adversarial Examplescited by 422
  20. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  21. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  22. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  23. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
  24. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  25. 2023Are aligned neural networks adversarially aligned?cited by 305
  26. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  27. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 284
  28. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
  29. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  30. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  31. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teamingcited by 161
  32. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  33. 2021A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generationcited by 137
  34. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 123
  35. 2019Assessing The Factual Accuracy of Generated Textcited by 103
  36. 2023Generating Benchmarks for Factuality Evaluation of Language Modelscited by 103

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 21; 2025: 37; 2026: 27.

85 papers.

Sort:

By area:

  1. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 146
  2. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 113
  3. 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documentscited by 93
  4. 2024EasyJailbreak: A Unified Framework for Jailbreaking Large Language Modelscited by 57
  5. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)cited by 54
  6. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  7. 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiescited by 36
  8. 2024Erasing Conceptual Knowledge from Language Modelscited by 34
  9. 2024AdvWave: Stealthy Adversarial Jailbreak Attack against Large Audio-Language Modelscited by 33
  10. 2025Adversarial Reasoning at Jailbreaking Timecited by 27
  11. 2024Mission Impossible: A Statistical Perspective on Jailbreaking LLMscited by 21
  12. 2024Adversarial Decoding: Generating Readable Documents for Adversarial Objectivescited by 16
38 more
  1. 2024Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilitiescited by 16
  2. 2024SequentialBreak: Large Language Models Can be Fooled by Embedding Jailbreak Prompts into Sequential Prompt Chainscited by 16
  3. 2025LLM-Safety Evaluations Lack Robustnesscited by 16
  4. 2025Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Textcited by 16
  5. 2025Fun-tuning: Characterizing the Vulnerability of Proprietary LLMs to Optimization-based Prompt Injection Attacks via the Fine-Tuning Interfacecited by 15
  6. 2025REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objectivecited by 15
  7. 2025STACK: Adversarial Attacks on LLM Safeguard Pipelinescited by 14
  8. 2025May I have your Attention? Breaking Fine-Tuning based Prompt Injection Defenses using Architecture-Aware Attackscited by 14
  9. 2025Short-length Adversarial Training Helps LLMs Defend Long-length Jailbreak Attacks: Theoretical and Empirical Evidencecited by 13
  10. 2024Adversarial Attacks on Large Language Models Using Regularized Relaxationcited by 12
  11. 2024Talking Nonsense: Probing Large Language Models' Understanding of Adversarial Gibberish Inputscited by 11
  12. 2025Jailbreak Attack Initializations as Extractors of Compliance Directionscited by 10
  13. 2025Single-pass Detection of Jailbreaking Input in Large Language Modelscited by 10
  14. 2025MixAT: Combining Continuous and Discrete Adversarial Training for LLMscited by 10
  15. 2026Under the Hood of SKILL.md: Semantic Supply-chain Attacks on AI Agent Skill Registrycited by 10
  16. 2024FLRT: Fluent Student-Teacher Redteamingcited by 9
  17. 2025SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinationscited by 9
  18. 2025Mitigating Indirect Prompt Injection via Instruction-Following Intent Analysiscited by 9
  19. 2024Testing the Limits of Jailbreaking Defenses with the Purple Problemcited by 8
  20. 2025Smoothed Embeddings for Robust Language Modelscited by 8
  21. 2025Universal Zero-shot Embedding Inversioncited by 8
  22. 2025LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasurescited by 8
  23. 2025Attacker's Noise Can Manipulate Your Audio-based LLM in the Real Worldcited by 8
  24. 2024Atoxia: Red-teaming Large Language Models with Target Toxic Answerscited by 7
  25. 2025Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attackscited by 7
  26. 2026Boundary Point Jailbreaking of Black-Box LLMscited by 7
  27. 2025Adversarial D'ej`a Vu: Jailbreak Dictionary Learning for Stronger Generalization to Unseen Attackscited by 6
  28. 2025AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Researchcited by 5
  29. 2026Now You Hear Me: Audio Narrative Attacks Against Large Audio-Language Modelscited by 5
  30. 2026Confundo: Learning to Generate Robust Poison for Practical RAG Systemscited by 5
  31. 2024Investigating Adversarial Trigger Transfer in Large Language Modelscited by 4
  32. 2025Targeting Alignment: Extracting Safety Classifiers of Aligned LLMscited by 3
  33. 2025Fast Proxies for LLM Robustness Evaluationcited by 3
  34. 2025Adversarial Attack on Large Language Models using Exponentiated Gradient Descentcited by 3
  35. 2025When Good Sounds Go Adversarial: Jailbreaking Audio-Language Models with Benign Inputscited by 3
  36. 2025Semantic Representation Attack against Aligned Large Language Modelscited by 3
  37. 2024New Emerged Security and Privacy of Pre-trained Model: a Survey and Outlookcited by 2
  38. 2025Jailbreaking with Universal Multi-Promptscited by 2

50 most cited of 85 All 85 (JSON)