mlsanityData to 14 Oct 2026

The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search

arXiv 2512.01353 ↗First posted 1 Dec 2025cs.CRcited by 7

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large language models (LLMs) remain vulnerable to jailbreak attacks that bypass safety guardrails to elicit harmful outputs. Existing approaches overwhelmingly operate within the prompt-optimization paradigm: whether through traditional algorithmic search or recent agent-based workflows, the resulting prompts typically retain malicious semantic signals that modern guardrails are primed to detect. In contrast, we identify a deeper, largely overlooked vulnerability stemming from the highly interconnected nature of an LLM's internal knowledge. This structure allows harmful objectives to be realized by weaving together sequences of benign sub-queries, each of which individually evades detection. To exploit this loophole, we introduce the Correlated Knowledge Attack Agent (CKA-Agent), a dynamic framework that reframes jailbreaking as an adaptive, tree-structured exploration of the target model's knowledge base. The CKA-Agent issues locally innocuous queries, uses model responses to guide exploration across multiple paths, and ultimately assembles the aggregated information to achieve the original harmful objective. Evaluated across state-of-the-art commercial LLMs (Gemini2.5-Flash/Pro, GPT-oss-120B, Claude-Haiku-4.5), CKA-Agent consistently achieves over 95% success rates even against strong guardrails, underscoring the severity of this vulnerability and the urgent need for defenses against such knowledge-decomposition attacks. Our codes are available at https://github.com/Graph-COM/CKA-Agent.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 54 of its 65 references that are in the map. First posted on arXiv 1 Dec 2025.

Builds on

54 of 65 references.

Closest ancestors · 27

Same direction, newest first

  1. 2025Adjacent Words, Divergent Intents: Jailbreaking Large Language Models via Task Concurrency24 Oct
  2. 2025MAJIC: Markovian Adaptive Jailbreaking via Iterative Composition of Diverse Innovative Strategies18 Aug
  3. 2025Alphabet Index Mapping: Jailbreaking LLMs through Semantic Dissimilarity15 Jun
  4. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents15 Apr
  5. 2025Prompt, Divide, and Conquer: Bypassing Large Language Model Safety Filters via Segmented and Distributed Prompt Processing27 Mar
  6. 2025Tempest: Autonomous Multi-Turn Jailbreaking of Large Language Models with Tree Search13 Mar
  7. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Models20 Oct
  8. 2024You Know What I'm Saying: Jailbreak Attack via Implicit Reference4 Oct
  9. 2024FlipAttack: Jailbreak LLMs via Flipping2 Oct
  10. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey5 Jul
  11. 2024When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search13 Jun
  12. 2024Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters30 May
15 more
  1. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
  2. 2024Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialogue27 Feb
  3. 2024A StrongREJECT for Empty Jailbreaks15 Feb
  4. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability13 Feb
  5. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  6. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  7. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization15 Nov
  8. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily14 Nov
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  10. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  11. 2023Low-Resource Languages Jailbreak GPT-43 Oct
  12. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  13. 2023FuzzLLM: A Novel and Universal Fuzzing Framework for Proactively Discovering Jailbreak Vulnerabilities in Large Language Models11 Sep
  14. 2023Detecting Language Model Attacks with Perplexity27 Aug
  15. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 27

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  4. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  5. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  6. 2022Red Teaming Language Models with Language Modelscited by 1,026
  7. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  8. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  9. 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
  10. 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
  11. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 167
  12. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 163
15 more
  1. 2025Qwen3Guard Technical Reportcited by 155
  2. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 90
  3. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 86
  4. 2024Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLMcited by 77
  5. 2024Automated Red Teaming with GOAT: the Generative Offensive Agent Testercited by 45
  6. 2024RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agentcited by 44
  7. 2025AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integrationcited by 43
  8. 2025Jailbreak-R1: Exploring the Jailbreak Capabilities of LLMs via Reinforcement Learningcited by 27
  9. 2024Diverse and Effective Red Teaming with Auto-generated Rewards and Multi-step Reinforcement Learningcited by 25
  10. 2025Eliciting Language Model Behaviors with Investigator Agentscited by 23
  11. 2024Evaluating Deep Unlearning in Large Language Modelscited by 14
  12. 2025Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awarenesscited by 14
  13. 2025CoP: Agentic Red-teaming for Large Language Models using Composition of Principlescited by 13
  14. 2024Ferret: Faster and Effective Automated Red Teaming with Reward-Based Scoring Techniquecited by 8
  15. 2025Strategize Globally, Adapt Locally: A Multi-Turn Red Teaming Agent with Dual-Level Learningcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026MoEEdit: Efficient and Routing-Stable Knowledge Editing for Mixture-of-Experts LLMscited by 4
  2. 2026D-Judge: Disrupting Multi-Turn Jailbreaks using Semantics-Preserving Output Rewritingcited by 1
  3. 2026Decomposition Attacks Across Unlinkable Identities: Limits of Stateful Defenses for LLM Servicescited by 1
  4. 2026One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intentcited by 0
  5. 2026Toward Self-Evolution-Ready Workflow Harnesses: A Reversible Migration Path and Convertibility Taxonomy for Expert LLM Pipelinescited by 0
  6. 2026SEAD: A State-Based Perspective on Attack and Defense in Tool-Using Agentscited by 0
  7. 2026Moral Combat: Exploiting Moral Foundations Theory to Jailbreak LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025GraphAttack: Exploiting Representational Blindspots in LLM Safety Mechanisms
  2. 2025Jailbreaking LLMs via Semantically Relevant Nested Scenarios with Targeted Toxic Knowledge
  3. 2026ATTCK-R: Knowledge-Aware Long-Horizon Jailbreaking for Protected LLMs
  4. 2026From Shield to Target: Denial-of-Service Attacks on LLM-Based Agent Guardrails
  5. 2024PathSeeker: Exploring LLM Security Vulnerabilities with a Reinforcement Learning-Based Jailbreak Approach
  6. 2025PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks
  7. 2025Forewarned is Forearmed: Pre-Synthesizing Jailbreak-like Instructions to Enhance LLM Safety Guardrail to Potential Attacks
  8. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystem