mlsanityData to 7 Oct 2026

Breaking the Assumptions: Auditing Input-Side Jailbreak Defenses Against Semantic Attacks

arXiv 2608.21895 ↗First posted 22 Aug 2026cs.CR cs.AIcited by 0

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Locally deployed Large Language Models (LLMs) via inference engines such as Ollama run without the moderation and abuse detection present in API-served models. Therefore, the safety of LLMs depends on the defense mechanisms used, and their effectiveness depends on the assumptions on which they were designed. This paper does an audit of defense mechanisms under jailbreak attacks on locally deployed models. Some defenses provide formal guarantees (SmoothLLM, Erase-and-Check, Sequential Monitors), while others rely on empirical detection results (Semantic Smoothing, Self-Denoised Smoothing, Perplexity Filtering). Instead of merely observing that defenses fail, we trace each failure back to the specific assumption: for every defense, we extract the condition it relies on, derive the empirical pattern a violation should produce, and test that prediction on six open-weight models (14B to 35B parameters) with a corpus of 100 jailbreak prompts taken from more than 40 public sources, totalling 13,800 evaluation records.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 14 of its 27 references that are in the map. First posted on arXiv 22 Aug 2026.

Builds on

14 of 27 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Advancing the Robustness of Large Language Models through Self-Denoised Smoothing18 Apr
  2. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack2 Apr
  3. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing25 Feb
  4. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  6. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  7. 2023Certifying LLM Safety against Adversarial Prompting6 Sep
  8. 2023Detecting Language Model Attacks with Perplexity27 Aug

Other references · 6

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  4. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  5. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  6. 2021Certified Robustness to Text Adversarial Attacks by Randomized [MASK]cited by 57

Built on by

Not cited yet.