mlsanityData to 7 Oct 2026

SAID: Safety-Aware Intent Defense via Prefix Probing for Large Language Models

arXiv 2510.20129 ↗First posted 23 Oct 2025cs.CR cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) remain vulnerable to jailbreak attacks, where adversarially crafted prompts induce policy-violating responses despite safety alignment. Existing defenses typically improve safety through external filtering, auxiliary guardrails, or decoding-time control. However, these interventions often reduce practical deployability because they may require additional model access, introduce extra inference cost, or affect benign-task utility. In this paper, we propose Safety-Aware Intent Defense (SAID), a training-free jailbreak defense framework based on intent-level safety probing. SAID first distills potentially obfuscated user inputs into concise core intents using the target model itself. It then applies a validated safety prefix to probe each distilled intent and elicit the model's safety-aware response. Finally, a conservative aggregation rule rejects the original request if any distilled intent is identified as unsafe. This design enables black-box-compatible defense without updating model parameters or modifying the decoding process. Experiments on four open-source LLMs under six representative jailbreak attacks show that SAID achieves state-of-the-art defense performance in reducing harmful responses while maintaining competitive utility on benign tasks. Further analyses on prefix variants, hierarchical distillation, and inference efficiency demonstrate that SAID provides a practical safety-utility trade-off for securing LLMs against jailbreak threats.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 24 of its 33 references that are in the map. First posted on arXiv 23 Oct 2025.

Builds on

24 of 33 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Intention Analysis Makes LLMs A Good Jailbreak Defender12 Jan
  2. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  3. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  4. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  5. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Models1 Sep
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 18

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  3. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  4. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  5. 2023Detecting Language Model Attacks with Perplexitycited by 411
  6. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  7. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 251
  8. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  9. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  10. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  11. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 103
  12. 2023Attack Prompt Generation for Red Teaming and Defending Large Language Modelscited by 73
6 more
  1. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
  2. 2024Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Cluescited by 57
  3. 2025JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulationcited by 46
  4. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Modelscited by 36
  5. 2024Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Modelscited by 23
  6. 2025ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attackscited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Defending Large Language Models Against Jailbreak Attacks via In-Decoding Safety-Awareness Probing
  2. 2025ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoning
  3. 2025Reasoned Safety Alignment: Ensuring Jailbreak Defense via Answer-Then-Check
  4. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing
  5. 2025Decoding One Safety Trigger Token for Balancing Safety and Usability in Large Language Models
  6. 2025Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval
  7. 2024Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attacks
  8. 2025Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation