mlsanityData to 7 Oct 2026

Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters

arXiv 2405.20413 ↗First posted 30 May 2024cs.CR cs.CL cs.CVcited by 39

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large Language Models (LLMs) are typically harmless but remain vulnerable to carefully crafted prompts known as ``jailbreaks'', which can bypass protective measures and induce harmful behavior. Recent advancements in LLMs have incorporated moderation guardrails that can filter outputs, which trigger processing errors for certain malicious questions. Existing red-teaming benchmarks often neglect to include questions that trigger moderation guardrails, making it difficult to evaluate jailbreak effectiveness. To address this issue, we introduce JAMBench, a harmful behavior benchmark designed to trigger and evaluate moderation guardrails. JAMBench involves 160 manually crafted instructions covering four major risk categories at multiple severity levels. Furthermore, we propose a jailbreak method, JAM (Jailbreak Against Moderation), designed to attack moderation guardrails using jailbreak prefixes to bypass input-level filters and a fine-tuned shadow model functionally equivalent to the guardrail model to generate cipher characters to bypass output-level filters. Our extensive experiments on four LLMs demonstrate that JAM achieves higher jailbreak success (~ × 19.88) and lower filtered-out rates (~ × 1/6) than baselines.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 28 of its 43 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

28 of 43 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models28 Mar
  2. 2024DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakers25 Feb
  3. 2024Query-Based Adversarial Prompt Generation19 Feb
  4. 2024GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Models5 Feb
  5. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization15 Nov
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  7. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  8. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  10. 2023Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Models17 Jul
  11. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 17

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  7. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  8. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  9. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  10. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  11. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  12. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
5 more
  1. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  2. 2023SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoningcited by 157
  3. 2017Deceiving Google's Perspective API Built for Detecting Toxic Commentscited by 99
  4. 2023In ChatGPT We Trust? Measuring and Characterizing the Reliability of ChatGPTcited by 35
  5. 2023AI in the Gray: Exploring Moderation Policies in Dialogic Large Language Models vs. Human Answers in Controversial Topicscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 20; 2026: 10.

39 papers.

Sort:

By area:

  1. 2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 132
  2. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  3. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
  4. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMscited by 47
  5. 2024JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Frameworkcited by 44
  6. 2025From Judgment to Interference: Early Stopping LLM Harmful Outputs via Streaming Content Monitoringcited by 20
  7. 2024Endless Jailbreaks with Bijection Learningcited by 18
  8. 2024You Know What I'm Saying: Jailbreak Attack via Implicit Referencecited by 13
  9. 2025Short-length Adversarial Training Helps LLMs Defend Long-length Jailbreak Attacks: Theoretical and Empirical Evidencecited by 13
  10. 2024Chain-of-Jailbreak Attack for Image Generation Models via Editing Step by Stepcited by 11
  11. 2026Obscure but Effective: Classical Chinese Jailbreak Prompt Optimization via Bio-Inspired Searchcited by 10
  12. 2025DualBreach: Efficient Dual-Jailbreaking via Target-Driven Initialization and Multi-Target Optimizationcited by 9
27 more
  1. 2025Evolving Security in LLMs: A Study of Jailbreak Attacks and Defensescited by 8
  2. 2025All Code, No Thought: Current Language Models Struggle to Reason in Ciphered Languagecited by 7
  3. 2025OMNIGUARD: An Efficient Approach for AI Safety Moderation Across Languages and Modalitiescited by 5
  4. 2025SafeLLM: Unlearning Harmful Outputs from Large Language Models against Jailbreak Attackscited by 4
  5. 2025Access Controls Will Solve the Dual-Use Dilemmacited by 3
  6. 2025Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulationcited by 3
  7. 2026Controlling Output Rankings in Generative Engines for LLM-based Searchcited by 3
  8. 2026SCI-Defense: Defending Manipulation Attacks from Generative Engine Optimizationcited by 3
  9. 2025From Hallucinations to Jailbreaks: Rethinking the Vulnerability of Large Foundation Modelscited by 2
  10. 2025InfoFlood: Jailbreaking Large Language Models with Information Overloadcited by 2
  11. 2025GuardVal: Dynamic Large Language Model Jailbreak Evaluation for Comprehensive Safety Testingcited by 2
  12. 2025MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulationcited by 2
  13. 2025PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploitscited by 2
  14. 2026Finding and Reactivating Post-Trained LLMs' Hidden Safety Mechanismscited by 2
  15. 2025MetaCipher: A Time-Persistent and Universal Multi-Agent Framework for Cipher-Based Jailbreak Attacks for LLMscited by 1
  16. 2025ASTRA: An Automated Framework for Strategy Discovery, Retrieval, and Evolution for Jailbreaking LLMscited by 1
  17. 2026Break Me If You Can: Self-Jailbreaking of Aligned LLMs via Lexical Insertion Promptingcited by 1
  18. 2025Concept Attractors in LLMs and their Applicationscited by 1
  19. 2026To Defend Against Cyber Attacks, We Must Teach AI Agents to Hackcited by 1
  20. 2026Understanding and Improving Continuous Adversarial Training for LLMs via In-context Learning Theorycited by 1
  21. 2024BiasJailbreak:Analyzing Ethical Biases and Jailbreak Vulnerabilities in Large Language Modelscited by 0
  22. 2025VLMs Can Aggregate Scattered Training Patchescited by 0
  23. 2025MEUV: Achieving Fine-Grained Capability Activation in Large Language Models via Mutually Exclusive Unlock Vectorscited by 0
  24. 2025Distillability of LLM Security Logic: Predicting Attack Success Rate of Outline Filling Attack via Ranking Regressioncited by 0
  25. 2026Structured Semantic Cloaking for Jailbreak Attacks on Large Language Modelscited by 0
  26. 2026AInterviewer: A Platform for Designing and Conducting AI-led Qualitative Interviewscited by 0
  27. 2026SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Modelscited by 0