mlsanityData to 14 Oct 2026

A Real-Time, Self-Tuning Moderator Framework for Adversarial Prompt Detection

arXiv 2508.07139 ↗First posted 10 Aug 2025cs.CR cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Ensuring LLM alignment is critical to information security as AI models become increasingly widespread and integrated in society. Unfortunately, many defenses against adversarial attacks and jailbreaking on LLMs cannot adapt quickly to new attacks, degrade model responses to benign prompts, or introduce significant barriers to scalable implementation. To mitigate these challenges, we introduce a real-time, self-tuning (RTST) moderator framework to defend against adversarial attacks while maintaining a lightweight training footprint. We empirically evaluate its effectiveness using Google's Gemini models against modern, effective jailbreaks. Our results demonstrate the advantages of an adaptive, minimally intrusive framework for jailbreak defense over traditional fine-tuning or classifier models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 31 of its 50 references that are in the map. First posted on arXiv 10 Aug 2025.

Builds on

31 of 50 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Models1 Sep
  2. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  3. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  3. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  4. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  5. 2023Detecting Language Model Attacks with Perplexitycited by 431
  6. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
  7. 2023Certifying LLM Safety against Adversarial Promptingcited by 266
  8. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 181
  9. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 170
  10. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 156
  11. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 145
  12. 2025GuardReasoner: Towards Reasoning-based LLM Safeguardscited by 100
15 more
  1. 2025LLM4SR: A Survey on Large Language Models for Scientific Researchcited by 83
  2. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 80
  3. 2025Safety in Large Reasoning Models: A Surveycited by 64
  4. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 63
  5. 2025Inverse Scaling in Test-Time Computecited by 50
  6. 2024An Early Categorization of Prompt Injection Attacks on Large Language Modelscited by 30
  7. 2025DarkMind: Latent Chain-of-Thought Backdoor in Customized LLMscited by 20
  8. 2024Automated Progressive Red Teamingcited by 13
  9. 2024Cognitive Overload Attack:Prompt Injection for Long Contextcited by 13
  10. 2025AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Securitycited by 10
  11. 2025From ASR to ASP: Evaluating Prompt Attack Vulnerabilities Against Open-Source LLMscited by 6
  12. 2024Programming with AI: Evaluating ChatGPT, Gemini, AlphaCode, and GitHub Copilot for Programmerscited by 5
  13. 2025Chain-of-Defensive-Thought: Structured Reasoning Elicits Robustness in Large Language Models against Reference Corruptioncited by 5
  14. 2024Generative AI Security: Challenges and Countermeasurescited by 4
  15. 2024[WIP] Jailbreak Paradox: The Achilles' Heel of LLMscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Fight Back Against Jailbreaking via Prompt Adversarial Tuning
  2. 2024Adversarial Tuning: Defending Against Jailbreak Attacks for LLMs
  3. 2025Efficient and Stealthy Jailbreak Attacks via Adversarial Prompt Distillation from LLMs to SLMs
  4. 2026The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoring
  5. 2025ShieldLearner: A New Paradigm for Jailbreak Attack Defense in LLMs
  6. 2025The TIP of the Iceberg: Revealing a Hidden Class of Task-in-Prompt Adversarial Attacks on LLMs
  7. 2025AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models
  8. 2025Adversarial Prompt Evaluation: Systematic Benchmarking of Guardrails Against Prompt Input Attacks on LLMs