mlsanityData to 7 Oct 2026

Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained Optimization

arXiv 2405.09113 ↗First posted 15 May 2024cs.LGcited by 16

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Recent research indicates that large language models (LLMs) are susceptible to jailbreaking attacks that can generate harmful content. This paper introduces a novel token-level attack method, Adaptive Dense-to-Sparse Constrained Optimization (ADC), which has been shown to successfully jailbreak multiple open-source LLMs. Drawing inspiration from the difficulties of discrete token optimization, our method relaxes the discrete jailbreak optimization into a continuous optimization process while gradually increasing the sparsity of the optimizing vectors. This technique effectively bridges the gap between discrete and continuous space optimization. Experimental results demonstrate that our method is more effective and efficient than state-of-the-art token-level methods. On Harmbench, our approach achieves the highest attack success rate on seven out of eight LLMs compared to the latest jailbreak methods. Trigger Warning: This paper contains model behavior that can be offensive in nature.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 36 of its 49 references that are in the map. First posted on arXiv 15 May 2024.

Builds on

36 of 49 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  3. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  5. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  7. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  8. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  9. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  10. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  11. 2022Red Teaming Language Models with Language Modelscited by 983
  12. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
19 more
  1. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  2. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  3. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  4. 2023Are aligned neural networks adversarially aligned?cited by 305
  5. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 288
  6. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  7. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  8. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  9. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  10. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 179
  11. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
  12. 2021Challenges in Detoxifying Language Modelscited by 169
  13. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 159
  14. 2023Fundamental Limitations of Alignment in Large Language Modelscited by 152
  15. 2023#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Modelscited by 108
  16. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 77
  17. 2022Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Modelscited by 67
  18. 2023Black Box Adversarial Prompting for Foundation Modelscited by 62
  19. 2024Learning or Self-aligning? Rethinking Instruction Fine-tuningcited by 43

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 9; 2026: 5.

16 papers.

Sort:

By area:

  1. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 112
  2. 2024Emoji Attack: Enhancing Jailbreak Attacks Against Judge LLM Detectioncited by 20
  3. 2025Jailbreak Attack Initializations as Extractors of Compliance Directionscited by 10
  4. 2026Obscure but Effective: Classical Chinese Jailbreak Prompt Optimization via Bio-Inspired Searchcited by 10
  5. 2025MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?cited by 5
  6. 2024Functional Homotopy: Smoothing Discrete Optimization via Continuous Parameters for LLM Jailbreak Attackscited by 4
  7. 2025On Surjectivity of Neural Networks: Can you elicit any behavior from your model?cited by 4
  8. 2025MIST: Jailbreaking Black-box Large Language Models via Iterative Semantic Tuningcited by 3
  9. 2025Clone What You Can't Steal: Black-Box LLM Replication via Logit Leakage and Distillationcited by 3
  10. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  11. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  12. 2025Jailbreak-Zero: A Path to Pareto Optimal Red Teaming for Large Language Modelscited by 1
4 more
  1. 2026TrailBlazer: History-Guided Reinforcement Learning for Black-Box LLM Jailbreakingcited by 1
  2. 2026Jailbreaking the Matrix: Nullspace Steering for Controlled Model Subversioncited by 0
  3. 2026Suan: Rectifying Direct Preference Safety Alignment in Large Language Modelscited by 0
  4. 2026Arbitrary Cipher Attacks Against Large Language Models Do Not Require Fine-Tuningcited by 0