mlsanityData to 7 Oct 2026

Detecting Fluent Optimization-Based Adversarial Prompts via Sequential Entropy Changes

arXiv 2605.19966 ↗First posted 19 May 2026cs.LG cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Optimization-based adversarial suffixes can jailbreak aligned large language models (LLMs) while remaining fluent, weakening static and windowed perplexity-based detectors. We cast adversarial suffix detection as an online change-point detection problem over the token-level next-token entropy stream. Using the LLM system prompt to estimate a robust baseline, we standardize user-token entropies and apply a one-sided CUSUM statistic. The resulting detector, CPD Online (CPD), is model-agnostic, training-free, runs online, and localizes the adversarial suffix onset. On a benchmark of 1,012 optimization-based suffix attacks (GCG, AutoDAN, AdvPrompter, BEAST, AutoDAN-HGA) and 1,012 perplexity-controlled benign prompts, CPD improves F1 over the strongest windowed-perplexity baseline on all six open-weight chat models (LLaMA-2-7B/13B, Vicuna-7B/13B, Qwen2.5-7B/14B). On LLaMA-2-7B at the canonical CUSUM setting (k=0), CPD reaches AUROC 0.88 and F1 0.82. Beyond prompt-level detection, CPD concentrates 79.6% of its triggers inside the adversarial suffix, versus 17-46% for windowed perplexity. Finally, when used as a lightweight gate for LLaMA Guard, CPD reduces guard calls by 17-22% on a high-volume, benign-dominated deployment while preserving guard-level detection quality

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 18 of its 33 references that are in the map. First posted on arXiv 19 May 2026.

Builds on

18 of 33 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Single-pass Detection of Jailbreaking Input in Large Language Models21 Feb
  2. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models23 Oct
  3. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  4. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Models1 Sep
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 13

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  4. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  5. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  6. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  7. 2023Detecting Language Model Attacks with Perplexitycited by 411
  8. 2023AI Alignment: A Comprehensive Surveycited by 256
  9. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  10. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 146
  11. 2024Fast Adversarial Attacks on Language Models In One GPU Minutecited by 85
  12. 2023Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Informationcited by 32
1 more
  1. 2015Scan B-Statistic for Kernel Change-Point Detectioncited by 28

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Unleashing the Unseen: Harnessing Benign Datasets for Jailbreaking Large Language Models
  2. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs
  3. 2024Advancing Adversarial Suffix Transfer Learning on Aligned Large Language Models
  4. 2025LatentBreak: Jailbreaking Large Language Models through Latent Space Feedback
  5. 2025Universal and Transferable Adversarial Attack on Large Language Models Using Exponentiated Gradient Descent
  6. 2024Human-Interpretable Adversarial Prompt Attack on Large Language Models with Situational Context
  7. 2024Towards Universal and Black-Box Query-Response Only Attack on LLMs with QROA
  8. 2025Universal Adversarial Suffixes for Language Models Using Reinforcement Learning with Calibrated Reward