mlsanityData to 7 Oct 2026

WordGame: Efficient & Effective LLM Jailbreak via Simultaneous Obfuscation in Query and Response

arXiv 2405.14023 ↗First posted 22 May 2024cs.LGcited by 28

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

The recent breakthrough in large language models (LLMs) such as ChatGPT has revolutionized production processes at an unprecedented pace. Alongside this progress also comes mounting concerns about LLMs' susceptibility to jailbreaking attacks, which leads to the generation of harmful or unsafe content. While safety alignment measures have been implemented in LLMs to mitigate existing jailbreak attempts and force them to become increasingly complicated, it is still far from perfect. In this paper, we analyze the common pattern of the current safety alignment and show that it is possible to exploit such patterns for jailbreaking attacks by simultaneous obfuscation in queries and responses. Specifically, we propose WordGame attack, which replaces malicious words with word games to break down the adversarial intent of a query and encourage benign content regarding the games to precede the anticipated harmful content in the response, creating a context that is hardly covered by any corpus used for safety alignment. Extensive experiments demonstrate that WordGame attack can break the guardrails of the current leading proprietary and open-source LLMs, including the latest Claude-3, GPT-4, and Llama-3 models. Further ablation studies on such simultaneous obfuscation in query and response provide evidence of the merits of the attack strategy beyond an individual attack.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 25 of its 55 references that are in the map. First posted on arXiv 22 May 2024.

Builds on

25 of 55 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakers25 Feb
  2. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedback19 Oct
  3. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM18 Sep
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  5. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 20

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2017Deep reinforcement learning from human preferencescited by 3,846
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  7. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  8. 2023LIMA: Less Is More for Alignmentcited by 1,080
  9. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  10. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  11. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  12. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
8 more
  1. 2023Detecting Language Model Attacks with Perplexitycited by 411
  2. 2023Large Language Models in Finance: A Surveycited by 223
  3. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  4. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  5. 2023Large Language Models in Law: A Surveycited by 128
  6. 2024Revolutionizing Finance with LLMs: An Overview of Applications and Insightscited by 88
  7. 2024Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Cluescited by 57
  8. 2023On the Safety of Open-Sourced Large Language Models: Does Alignment Really Prevent Them From Being Misused?cited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 15; 2026: 6.

28 papers.

Sort:

By area:

  1. 2024Robust LLM safeguarding via refusal feature adversarial trainingcited by 60
  2. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  3. 2024Mind the Inconspicuous: Revealing the Hidden Weakness in Aligned LLMs' Refusal Boundariescited by 26
  4. 2025Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarkscited by 26
  5. 2025EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safetycited by 21
  6. 2025X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usabilitycited by 20
  7. 2024Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAIcited by 14
  8. 2025Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Modelscited by 13
  9. 2024Recent advancements in LLM Red-Teaming: Techniques, Defenses, and Ethical Considerationscited by 8
  10. 2025Anyone Can Jailbreak: Prompt-Based Attacks on LLMs and T2Iscited by 6
  11. 2025Dagger Behind Smile: Fool LLMs with a Happy Ending Storycited by 5
  12. 2025Prompt, Divide, and Conquer: Bypassing Large Language Model Safety Filters via Segmented and Distributed Prompt Processingcited by 5
16 more
  1. 2025Toward a Dynamic Stackelberg Game-Theoretic Framework for Agentic AI Defense Against LLM Jailbreakingcited by 5
  2. 2025sudoLLM: On Multi-role Alignment of Language Modelscited by 4
  3. 2024Preventing Jailbreak Prompts as Malicious Tools for Cybercriminals: A Cyber Defense Perspectivecited by 3
  4. 2025Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulationcited by 3
  5. 2026Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks from Internal Representations of Large Language Modelscited by 3
  6. 2025Jailbreaking Large Language Models Through Content Concretizationcited by 2
  7. 2026LLM-enabled Applications Require System-Level Threat Monitoringcited by 2
  8. 2026Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarkingcited by 2
  9. 2025Advancing Jailbreak Strategies: A Hybrid Approach to Exploiting LLM Vulnerabilities and Bypassing Modern Defensescited by 1
  10. 2025ObfusQAte: A Proposed Framework to Evaluate LLM Robustness on Obfuscated Factual Question Answeringcited by 1
  11. 2026Jailbreaking Large Language Models through Iterative Tool-Disguised Attacks via Reinforcement Learningcited by 1
  12. 2024Knowledge Return Oriented Prompting (KROP)cited by 0
  13. 2025LeakSealer: A Semisupervised Defense for LLMs Against Prompt Injection and Leakage Attackscited by 0
  14. 2025Pluralistic Behavior Suite: Stress-Testing Multi-Turn Adherence to Custom Behavioral Policiescited by 0
  15. 2026Structured Semantic Cloaking for Jailbreak Attacks on Large Language Modelscited by 0
  16. 2026Who Judges the Judges? A Chinese Safety QA Benchmark for Evaluating LLM Responses and Safety Judgescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey
  2. 2024Playing Language Game with LLMs Leads to Jailbreaking
  3. 2025Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers
  4. 2024Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attacks
  5. 2025Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures
  6. 2025Jailbreaking Attacks vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race?
  7. 2025Do Methods to Jailbreak and Defend LLMs Generalize Across Languages?
  8. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs