mlsanityData to 14 Oct 2026

Accelerating Greedy Coordinate Gradient and General Prompt Optimization via Probe Sampling

arXiv 2403.01251 ↗First posted 2 Mar 2024cs.CLcited by 37

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Safety of Large Language Models (LLMs) has become a critical issue given their rapid progresses. Greedy Coordinate Gradient (GCG) is shown to be effective in constructing adversarial prompts to break the aligned LLMs, but optimization of GCG is time-consuming. To reduce the time cost of GCG and enable more comprehensive studies of LLM safety, in this work, we study a new algorithm called textttProbe sampling. At the core of the algorithm is a mechanism that dynamically determines how similar a smaller draft model's predictions are to the target model's predictions for prompt candidates. When the target model is similar to the draft model, we rely heavily on the draft model to filter out a large number of potential prompt candidates. Probe sampling achieves up to 5.6 times speedup using Llama2-7b-chat and leads to equal or improved attack success rate (ASR) on the AdvBench. Furthermore, probe sampling is also able to accelerate other prompt optimization techniques and adversarial methods, leading to acceleration of 1.8× for AutoPrompt, 2.4× for APE and 2.4× for AutoDAN.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 55 of its 70 references that are in the map. First posted on arXiv 2 Mar 2024.

Builds on

55 of 70 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Attacking Large Language Models with Projected Gradient Descent14 Feb
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  3. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 51

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
38 more
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  4. 2020Learning to summarize from human feedbackcited by 2,588
  5. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  6. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  7. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  8. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  9. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  10. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  11. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  12. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  13. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  14. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  15. 2024TinyLlama: An Open-Source Small Language Modelcited by 618
  16. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 605
  17. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 569
  18. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 565
  19. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  20. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 467
  21. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
  22. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 415
  23. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  24. 2021Pay Attention to MLPscited by 368
  25. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
  26. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 350
  27. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 336
  28. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 273
  29. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 234
  30. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 195
  31. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 188
  32. 2019The Evolved Transformercited by 180
  33. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 145
  34. 2023Auditing large language models: a three-layered approachcited by 137
  35. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 128
  36. 2023Prompt Optimization via Adversarial In-Context Learningcited by 33
  37. 2023Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Gamecited by 27
  38. 2023Align on the Fly: Adapting Chatbot Behavior to Established Normscited by 23

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 18; 2026: 10.

37 papers.

Sort:

By area:

  1. 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 110
  2. 2024MRJ-Agent: An Effective Jailbreak Agent for Multi-Round Dialoguecited by 33
  3. 2024Faster-GCG: Efficient Discrete Optimization Jailbreak Attacks against Aligned Large Language Modelscited by 25
  4. 2026MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safetycited by 21
  5. 2024Endless Jailbreaks with Bijection Learningcited by 19
  6. 2024Exploiting the Index Gradients for Optimization-Based Jailbreaking on Large Language Modelscited by 15
  7. 2025May I have your Attention? Breaking Fine-Tuning based Prompt Injection Defenses using Architecture-Aware Attackscited by 15
  8. 2024Trojan Detection in Large Language Models: Insights from The Trojan Detection Challengecited by 14
  9. 2025Manipulating LLM Web Agents with Indirect Prompt Injection Attack via HTML Accessibility Treecited by 14
  10. 2024GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMscited by 12
  11. 2025Jailbreak Attack Initializations as Extractors of Compliance Directionscited by 10
  12. 2025Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Modelscited by 10
25 more
  1. 2025SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinationscited by 9
  2. 2025A Survey of Automatic Prompt Optimization with Instruction-focused Heuristic-based Search Algorithmcited by 6
  3. 2025Mask-GCG: Are All Tokens in Adversarial Suffixes Necessary for Jailbreak Attacks?cited by 6
  4. 2025Mind the Web: The Security of Web Use Agentscited by 5
  5. 2024Layer-wise Alignment: Examining Safety Alignment Across Image Encoder Layers in Vision Language Modelscited by 4
  6. 2025Targeting Alignment: Extracting Safety Classifiers of Aligned LLMscited by 3
  7. 2026Intent Laundering: AI Safety Datasets Are Not What They Seemcited by 3
  8. 2026Evolving Skill-Structured Attack Memory Enhances LLM Jailbreakingcited by 3
  9. 2025Disrupting Vision-Language Model-Driven Navigation Services via Adversarial Object Fusioncited by 2
  10. 2025GPT, But Backwards: Exactly Inverting Language Model Outputscited by 2
  11. 2025Memories Retrieved from Many Paths: A Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Modelscited by 2
  12. 2026TAO-Attack: Toward Advanced Optimization-Based Jailbreak Attacks for Large Language Modelscited by 2
  13. 2026LPDS: Evaluating LLM Robustness Through Logic-Preserving Difficulty Scalingcited by 2
  14. 2026SlotGCG: Exploiting the Positional Vulnerability in LLMs for Jailbreak Attackscited by 2
  15. 2025ASTRA: An Automated Framework for Strategy Discovery, Retrieval, and Evolution for Jailbreaking LLMscited by 1
  16. 2025SteganoBackdoor: Evading Data-Poisoning Defenses via Steganographic Backdoorscited by 1
  17. 2026Latent-space Attacks for Refusal Evasion in Language Modelscited by 1
  18. 2026LaCache: Robust Semantic Caching for LLM Servingcited by 1
  19. 2024Poison Attacks and Adversarial Prompts Against an Informed University Virtual Assistantcited by 0
  20. 2025Inverse Language Modeling towards Robust and Grounded LLMscited by 0
  21. 2025ELPO: Ensemble Learning Based Prompt Optimization for Large Language Modelscited by 0
  22. 2025TASO: Jailbreak LLMs via Alternative Template and Suffix Optimizationcited by 0
  23. 2025Safe2Harm: Semantic Isomorphism Attacks for Jailbreaking Large Language Modelscited by 0
  24. 2026TrapSuffix: Proactive Defense Against Adversarial Suffixes in Jailbreakingcited by 0
  25. 2026The Safeguard Worked. Is the LLM System Safer?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Resurgence of GCG Adversarial Attacks on Large Language Models
  2. 2025Sampling-aware Adversarial Attacks Against Large Language Models
  3. 2024Boosting Jailbreak Attack with Momentum
  4. 2025GCG Attack On A Diffusion LLM
  5. 2026Bypassing Prompt Injection Detectors through Evasive Injections
  6. 2026RECAP: A Resource-Efficient Method for Adversarial Prompting in Large Language Models
  7. 2026Exposing Long-Tail Safety Failures in Large Language Models through Efficient Diverse Response Sampling
  8. 2025Safeguarding Efficacy in Large Language Models: Evaluating Resistance to Human-Written and Algorithmic Adversarial Prompts