Exploring and Developing a Pre-Model Safeguard with Draft Models
On the map: LLM safety and security → Jailbreaking
Abstract (arXiv)
Large Language Model (LLM) alignment remains vulnerable to jailbreak attacks that elicit unsafe responses, motivating pre-model and post-model guards. Pre-model guards audit the safety of prompts before invoking target models. However, relying solely on the prompt often leads to high false-negative rates (i.e., jailbreak attacks go undetected). Post-model guards address this issue by auditing both the user prompt and the target model's response. However, they incur a high computational cost, including increased token usage and processing time, because they operate after target model inference. In this paper, we introduce a safeguard design that leverages the transferability of jailbreak attacks to enforce prompt safety before target model inference. We first conduct a systematic study of jailbreak transferability, particularly from LLMs to small language models (SLMs). Through these experiments, we identify key factors influencing transferability. Building on these insights, we observe that responses from smaller draft models reflect the safety implications of those from large target models; ie given a jailbreak prompt constructed for an LLM, an SLM is likely to be triggered to generate an unaligned response. Based on this observation, our safeguard design leverages speculative inference with SLMs to generate a set of draft responses. It then feeds the original prompt and these drafts into existing guards to predict their safety. We demonstrate that this design reduces the false-negative rate of pre-model guards and offers a low Efficiency alternative to post-model guards. textcolorredbf Notice: This paper contains examples of harmful language.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 37 of its 62 references that are in the map. First posted on arXiv 19 May 2026.
Builds on
37 of 62 references.
Closest ancestors · 15
Same direction, newest first
- 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing28 May
- 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models28 Mar
- 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction28 Feb
- 2024Fast Adversarial Attacks on Language Models In One GPU Minute23 Feb
- 2024GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis21 Feb
- 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization15 Nov
- 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
- 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
- 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
- 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
- 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
- 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
3 more
Other references · 22
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
- 2019The Curious Case of Neural Text Degenerationcited by 2,881
- 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
- 2023LIMA: Less Is More for Alignmentcited by 1,080
- 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
- 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
- 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
10 more
- 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
- 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 251
- 2017Beam Search Strategies for Neural Machine Translationcited by 232
- 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
- 2023Online Speculative Decodingcited by 109
- 2021Early Stopping for Deep Image Priorcited by 44
- 2024Speculative Streaming: Fast LLM Inference without Auxiliary Modelscited by 37
- 2025ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrailscited by 29
- 2024Exploring the Deceptive Power of LLM-Generated Fake News: A Study of Real-World Detection Challengescited by 28
- 2025Bypassing Prompt Guards in Production with Controlled-Release Promptingcited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Beyond the Prompt: Jailbreaking Function-Calling LLMs via Simulated Moderation Traces
- 2025One Model Transfer to All: On Robust Jailbreak Prompts Generation against LLMs
- 2026GUARD-SLM: Token Activation-Based Defense Against Jailbreak Attacks for Small Language Models
- 2025PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks
- 2024Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attacks
- 2025Jailbreaking Attacks vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race?
- 2024Model-Editing-Based Jailbreak against Safety-aligned Large Language Models
- 2024Pruning for Protection: Increasing Jailbreak Resistance in Aligned LLMs Without Fine-Tuning