Jailbreaking Large Language Models Through Content Concretization
On the map: LLM safety and security → Jailbreaking
Abstract (arXiv)
Large Language Models (LLMs) are increasingly deployed for task automation and content generation, yet their safety mechanisms remain vulnerable to circumvention through different jailbreaking techniques. In this paper, we introduce textitContent Concretization (CC), a novel jailbreaking technique that iteratively transforms abstract malicious requests into concrete, executable implementations. CC is a two-stage process: first, generating initial LLM responses using lower-tier, less constrained safety filters models, then refining them through higher-tier models that process both the preliminary output and original prompt. We evaluate our technique using 350 cybersecurity-specific prompts, demonstrating substantial improvements in jailbreak Success Rates (SRs), increasing from 7% (no refinements) to 62% after three refinement iterations, while maintaining a cost of 7.5textcent~per prompt. Comparative A/B testing across nine different LLM evaluators confirms that outputs from additional refinement steps are consistently rated as more malicious and technically superior. Moreover, manual code analysis reveals that generated outputs execute with minimal modification, although optimal deployment typically requires target-specific fine-tuning. With eventual improved harmful code generation, these results highlight critical vulnerabilities in current LLM safety frameworks.
Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 12 of its 19 references that are in the map. First posted on arXiv 16 Sep 2025.
Builds on
12 of 19 references.
Closest ancestors · 11
Same direction, newest first
- 2025Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversion20 May
- 2025The Jailbreak Tax: How Useful are Your Jailbreak Outputs?14 Apr
- 2025Prompt, Divide, and Conquer: Bypassing Large Language Model Safety Filters via Segmented and Distributed Prompt Processing27 Mar
- 2025Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions6 Feb
- 2025CySecBench: Generative AI-based CyberSecurity-focused Prompt Dataset for Benchmarking Large Language Models2 Jan
- 2024SequentialBreak: Large Language Models Can be Fooled by Embedding Jailbreak Prompts into Sequential Prompt Chains10 Nov
- 2024WordGame: Efficient & Effective LLM Jailbreak via Simultaneous Obfuscation in Query and Response22 May
- 2024When "Competency" in Reasoning Opens the Door to Vulnerability: Jailbreaking LLMs via Novel Complex Ciphers16 Feb
- 2024Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues14 Feb
- 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability13 Feb
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
Other references · 1
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models
- 2025Stand on The Shoulders of Giants: Building JailExpert from Previous Attack Experience
- 2024Distract Large Language Models for Automatic Jailbreak Attack
- 2024Rethinking How to Evaluate Language Model Jailbreak
- 2025Jailbreaking in the Haystack
- 2025SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention
- 2024Stealthy Jailbreak Attacks on Large Language Models via Benign Data Mirroring
- 2023Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks