Self-Instruct Few-Shot Jailbreaking: Decompose the Attack into Pattern and Behavior Learning
On the map: Synthetic data and self-improvement for LLMs → Safety of self-evolving systems
also in 1 more
Abstract (arXiv)
Recently, several works have been conducted on jailbreaking Large Language Models (LLMs) with few-shot malicious demos. In particular, Zheng et al. focus on improving the efficiency of Few-Shot Jailbreaking (FSJ) by injecting special tokens into the demos and employing demo-level random search, known as Improved Few-Shot Jailbreaking (I-FSJ). Nevertheless, we notice that this method may still require a long context to jailbreak advanced models e.g. 32 shots of demos for Meta-Llama-3-8B-Instruct (Llama-3) citellama3modelcard. In this paper, we discuss the limitations of I-FSJ and propose Self-Instruct Few-Shot Jailbreaking (Self-Instruct-FSJ) facilitated with the demo-level greedy search. This framework decomposes the FSJ attack into pattern and behavior learning to exploit the model's vulnerabilities in a more generalized and efficient way. We conduct elaborate experiments to evaluate our method on common open-source models and compare it with baseline algorithms. Our code is available at https://github.com/iphosi/Self-Instruct-FSJ.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Safety of self-evolving systems, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 39 of its 55 references that are in the map. First posted on arXiv 14 Jan 2025.
Builds on
39 of 55 references.
References · 39
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2023Mistral 7Bcited by 3,164
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2024Qwen2 Technical Reportcited by 2,120
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
- 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
- 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
- 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
27 more
- 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
- 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
- 2023Detecting Language Model Attacks with Perplexitycited by 431
- 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
- 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 423
- 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
- 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
- 2023Low-Resource Languages Jailbreak GPT-4cited by 309
- 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 297
- 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 252
- 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 244
- 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 240
- 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 216
- 2024Foundational Challenges in Assuring Alignment and Safety of Large Language Modelscited by 199
- 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 195
- 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 170
- 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 157
- 2024Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuningcited by 138
- 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 130
- 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 128
- 2023Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Modelscited by 110
- 2024CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completioncited by 76
- 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defensescited by 70
- 2023Stealthy and Persistent Unalignment on Large Language Models via Backdoor Injectionscited by 39
- 2024Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!cited by 37
- 2023Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignmentcited by 30
- 2024Diversity Helps Jailbreak Large Language Modelscited by 10
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Distract Large Language Models for Automatic Jailbreak Attack
- 2025LLM Jailbreak Detection for (Almost) Free!
- 2025Rewrite to Jailbreak: Discover Learnable and Transferable Implicit Harmfulness Instruction
- 2025Guiding not Forcing: Enhancing the Transferability of Jailbreaking Attacks on LLMs via Removing Superfluous Constraints
- 2024JailPO: A Novel Black-box Jailbreak Framework via Preference Optimization against Aligned LLMs
- 2025Stand on The Shoulders of Giants: Building JailExpert from Previous Attack Experience
- 2025Advancing Jailbreak Strategies: A Hybrid Approach to Exploiting LLM Vulnerabilities and Bypassing Modern Defenses
- 2023Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks