mlsanityData to 14 Oct 2026

STShield: Single-Token Sentinel for Real-Time Jailbreak Detection in Large Language Models

arXiv 2503.17932 ↗First posted 23 Mar 2025cs.CL cs.AI cs.CRcited by 3

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large Language Models (LLMs) have become increasingly vulnerable to jailbreak attacks that circumvent their safety mechanisms. While existing defense methods either suffer from adaptive attacks or require computationally expensive auxiliary models, we present STShield, a lightweight framework for real-time jailbroken judgement. STShield introduces a novel single-token sentinel mechanism that appends a binary safety indicator to the model's response sequence, leveraging the LLM's own alignment capabilities for detection. Our framework combines supervised fine-tuning on normal prompts with adversarial training using embedding-space perturbations, achieving robust detection while preserving model utility. Extensive experiments demonstrate that STShield successfully defends against various jailbreak attacks, while maintaining the model's performance on legitimate queries. Compared to existing approaches, STShield achieves superior defense performance with minimal computational overhead, making it a practical solution for real-world LLM deployment.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 37 of its 53 references that are in the map. First posted on arXiv 23 Mar 2025.

Builds on

37 of 53 references.

Closest ancestors · 28

Same direction, newest first

  1. 2024HSF: Defending against Jailbreak Attacks with Hidden State Filtering31 Aug
  2. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs13 Jun
  3. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner8 Jun
  4. 2024Adversarial Tuning: Defending Against Jailbreak Attacks for LLMs7 Jun
  5. 2024Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attacks30 May
  6. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing28 May
  7. 2024Efficient Adversarial Training in LLMs with Continuous Attacks24 May
  8. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs11 Apr
  9. 2024Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledge8 Apr
  10. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
  11. 2024Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes1 Mar
  12. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing25 Feb
16 more
  1. 2024Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues14 Feb
  2. 2024Fight Back Against Jailbreaking via Prompt Adversarial Tuning9 Feb
  3. 2024JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMs8 Feb
  4. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks30 Jan
  5. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  6. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization15 Nov
  7. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  8. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  9. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  10. 2023Low-Resource Languages Jailbreak GPT-43 Oct
  11. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM18 Sep
  12. 2023Certifying LLM Safety against Adversarial Prompting6 Sep
  13. 2023Detecting Language Model Attacks with Perplexity27 Aug
  14. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  15. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
  16. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study23 May

Other references · 9

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  5. 2022Red Teaming Language Models with Language Modelscited by 1,026
  6. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  7. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  8. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 244
  9. 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contextscited by 94

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025SoK: Evaluating Jailbreak Guardrails for Large Language Modelscited by 21
  2. 2025IP Leakage Attacks Targeting LLM-Based Multi-Agent Systemscited by 8
  3. 2025Beyond Surface-Level Detection: Towards Cognitive-Driven Defense Against Jailbreak Attacks via Meta-Operations Reasoningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Test-Time Immunization: A Universal Defense Framework Against Jailbreaks for (Multimodal) Large Language Models
  2. 2024Prefix Guidance: A Steering Wheel for Large Language Models to Defend Against Jailbreak Attacks
  3. 2025Single-pass Detection of Jailbreaking Input in Large Language Models
  4. 2025JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulation
  5. 2026Causal Front-Door Adjustment for Robust Jailbreak Attacks on LLMs
  6. 2024Rapid Response: Mitigating LLM Jailbreaks with a Few Examples
  7. 2026SelfGrader: LLM Jailbreak Detection via Anchored Token-Level Logits
  8. 2026Sparse Autoencoders are Capable LLM Jailbreak Mitigators