# When "Competency" in Reasoning Opens the Door to Vulnerability: Jailbreaking LLMs via Novel Complex Ciphers

arXiv: https://arxiv.org/abs/2402.10601 · HTML: https://mlsanity.com/paper/2402.10601/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 25 of 25)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models](https://mlsanity.com/paper/2310.04451/) (2023)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [Baseline Defenses for Adversarial Attacks Against Aligned Language Models](https://mlsanity.com/paper/2309.00614/) (2023)
- [How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs](https://mlsanity.com/paper/2401.06373/) (2024)
- [GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher](https://mlsanity.com/paper/2308.06463/) (2023)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [Low-Resource Languages Jailbreak GPT-4](https://mlsanity.com/paper/2310.02446/) (2023)
- [Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks](https://mlsanity.com/paper/2302.05733/) (2023)
- [Multilingual Jailbreak Challenges in Large Language Models](https://mlsanity.com/paper/2310.06474/) (2023)
- [Automatically Auditing Large Language Models via Discrete Optimization](https://mlsanity.com/paper/2303.04381/) (2023)
- [Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve](https://mlsanity.com/paper/2309.13638/) (2023)
- [MART: Improving LLM Safety with Multi-round Automatic Red-Teaming](https://mlsanity.com/paper/2311.07689/) (2023)
- [Open Sesame! Universal Black Box Jailbreaking of Large Language Models](https://mlsanity.com/paper/2309.01446/) (2023)
- [Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information](https://mlsanity.com/paper/2311.11509/) (2023)
- [Summon a Demon and Bind it: A Grounded Theory of LLM Red Teaming](https://mlsanity.com/paper/2311.06237/) (2023)

## Built on by (most cited first, 30 of 30)

- [Jailbreak Attacks and Defenses Against Large Language Models: A Survey](https://mlsanity.com/paper/2407.04295/) (2024)
- [LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet](https://mlsanity.com/paper/2408.15221/) (2024)
- [Defending LLMs against Jailbreaking Attacks via Backtranslation](https://mlsanity.com/paper/2402.16459/) (2024)
- [JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models](https://mlsanity.com/paper/2407.01599/) (2024)
- [Jailbreak Large Vision-Language Models Through Multi-Modal Linkage](https://mlsanity.com/paper/2412.00473/) (2024)
- [Large Language Model Safety: A Holistic Survey](https://mlsanity.com/paper/2412.17686/) (2024)
- [Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)](https://mlsanity.com/paper/2407.14937/) (2024)
- [QA-LIGN: Aligning LLMs through Constitutionally Decomposed QA](https://mlsanity.com/paper/2506.08123/) (2025)
- [A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models](https://mlsanity.com/paper/2509.03871/) (2025)
- [Emerging Vulnerabilities in Frontier Models: Multi-Turn Jailbreak Attacks](https://mlsanity.com/paper/2409.00137/) (2024)
- [AutoJailbreak: Exploring Jailbreak Attacks and Defenses through a Dependency Lens](https://mlsanity.com/paper/2406.03805/) (2024)
- [Cognitive Overload Attack:Prompt Injection for Long Context](https://mlsanity.com/paper/2410.11272/) (2024)
- [GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methods](https://mlsanity.com/paper/2502.16903/) (2025)
- [CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenges](https://mlsanity.com/paper/2504.19093/) (2025)
- [Three Minds, One Legend: Jailbreak Large Reasoning Model with Adaptive Stacked Ciphers](https://mlsanity.com/paper/2505.16241/) (2025)
- [All Code, No Thought: Current Language Models Struggle to Reason in Ciphered Language](https://mlsanity.com/paper/2510.09714/) (2025)
- [Attention Eclipse: Manipulating Attention to Bypass LLM Safety-Alignment](https://mlsanity.com/paper/2502.15334/) (2025)
- [Should LLM Safety Be More Than Refusing Harmful Instructions?](https://mlsanity.com/paper/2506.02442/) (2025)
- [Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Models](https://mlsanity.com/paper/2506.16760/) (2025)
- [ASTRA: Autonomous Spatial-Temporal Red-teaming for AI Software Assistants](https://mlsanity.com/paper/2508.03936/) (2025)
- [Benchmarking Large Language Models for Cryptanalysis and Side-Channel Vulnerabilities](https://mlsanity.com/paper/2505.24621/) (2025)
- [AISA: Awakening Intrinsic Safety Awareness in Large Language Models against Jailbreak Attacks](https://mlsanity.com/paper/2602.13547/) (2026)
- [Intent Laundering: AI Safety Datasets Are Not What They Seem](https://mlsanity.com/paper/2602.16729/) (2026)
- [StructTransform: A Scalable Attack Surface for Safety-Aligned Large Language Models](https://mlsanity.com/paper/2502.11853/) (2025)
- [Jailbreaking Large Language Models Through Content Concretization](https://mlsanity.com/paper/2509.12937/) (2025)
- [Invisible Safety Threat: Malicious Finetuning for LLM via Steganography](https://mlsanity.com/paper/2603.08104/) (2026)
- [TestDG: Test-time Domain Generalization for Continual Test-time Adaptation](https://mlsanity.com/paper/2504.04981/) (2025)
- [OptAgent: Optimizing Query Rewriting for E-commerce via Multi-Agent Simulation](https://mlsanity.com/paper/2510.03771/) (2025)
- [RoguePrompt: Dual-Layer Encoding for Self-Reconstruction to Circumvent LLM Moderation](https://mlsanity.com/paper/2607.27373/) (2026)
- [Narrative Paradoxes in LLM Inference: Shaping Reasoning Trajectories toward Misalignment](https://mlsanity.com/paper/2610.16312/) (2026)

## Abstract (arXiv)

Recent advancements in Large Language Model (LLM) safety have primarily focused on mitigating attacks crafted in natural language or common ciphers (e.g. Base64), which are likely integrated into newer models' safety training. However, we reveal a paradoxical vulnerability: as LLMs advance in reasoning, they inadvertently become more susceptible to novel jailbreaking attacks. Enhanced reasoning enables LLMs to interpret complex instructions and decode complex user-defined ciphers, creating an exploitable security gap. To study this vulnerability, we introduce Attacks using Custom Encryptions (ACE), a jailbreaking technique that encodes malicious queries with novel ciphers. Extending ACE, we introduce Layered Attacks using Custom Encryptions (LACE), which applies multi-layer ciphers to amplify attack complexity. Furthermore, we develop CipherBench, a benchmark designed to evaluate LLMs' accuracy in decoding encrypted benign text. Our experiments reveal a critical trade-off: LLMs that are more capable of decoding ciphers are more vulnerable to LACE, with success rates on gpt-oss-20b escalating from 60% under ACE to 72% with LACE. These findings highlight a critical insight: as LLMs become more adept at deciphering complex user ciphers--many of which cannot be preemptively included in safety training--they become increasingly exploitable.