mlsanityData to 14 Oct 2026

Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues

arXiv 2402.09091 ↗First posted 14 Feb 2024cs.CR cs.AI cs.HCcited by 68

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

With the development of LLMs, the security threats of LLMs are getting more and more attention. Numerous jailbreak attacks have been proposed to assess the security defense of LLMs. Current jailbreak attacks primarily utilize scenario camouflage techniques. However their explicitly mention of malicious intent will be easily recognized and defended by LLMs. In this paper, we propose an indirect jailbreak attack approach, Puzzler, which can bypass the LLM's defense strategy and obtain malicious response by implicitly providing LLMs with some clues about the original malicious query. In addition, inspired by the wisdom of "When unable to attack, defend" from Sun Tzu's Art of War, we adopt a defensive stance to gather clues about the original malicious query through LLMs. Extensive experimental results show that Puzzler achieves a query success rate of 96.6% on closed-source LLMs, which is 57.9%-82.7% higher than baselines. Furthermore, when tested against the state-of-the-art jailbreak detection approaches, Puzzler proves to be more effective at evading detection compared to baselines.

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 19 of its 30 references that are in the map. First posted on arXiv 14 Feb 2024.

Builds on

19 of 30 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Pandora: Jailbreak GPTs by Retrieval Augmented Generation Poisoning13 Feb
  2. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  3. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily14 Nov
  4. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  6. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  7. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  9. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
  10. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study23 May

Other references · 9

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2023LIMA: Less Is More for Alignmentcited by 1,100
  4. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  5. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  6. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  7. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  8. 2023Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERTcited by 155
  9. 2023Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulnesscited by 130

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 23; 2025: 32; 2026: 13.

68 papers.

Sort:

By area:

  1. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
  2. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 173
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  4. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 88
  5. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 63
  6. 2024SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Typescited by 60
  7. 2025JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulationcited by 54
  8. 2024Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledgecited by 51
  9. 2024When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Searchcited by 47
  10. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 31
  11. 2024Glitch Tokens in Large Language Models: Categorization Taxonomy and Effective Detectioncited by 29
  12. 2024WordGame: Efficient & Effective LLM Jailbreak via Simultaneous Obfuscation in Query and Responsecited by 29
38 more
  1. 2025NeuroStrike: Neuron-Level Attacks on Aligned LLMscited by 22
  2. 2024PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMscited by 21
  3. 2025EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safetycited by 21
  4. 2025SoK: Evaluating Jailbreak Guardrails for Large Language Modelscited by 21
  5. 2024SequentialBreak: Large Language Models Can be Fooled by Embedding Jailbreak Prompts into Sequential Prompt Chainscited by 19
  6. 2024Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAIcited by 16
  7. 2025A Survey of Attacks on Large Language Modelscited by 15
  8. 2024AutoJailbreak: Exploring Jailbreak Attacks and Defenses through a Dependency Lenscited by 14
  9. 2025Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulationcited by 14
  10. 2024Efficient Detection of Toxic Prompts in Large Language Modelscited by 13
  11. 2024Playing Language Game with LLMs Leads to Jailbreakingcited by 11
  12. 2024SATA: A Paradigm for LLM Jailbreak via Simple Assistive Task Linkagecited by 11
  13. 2025Think Twice, Generate Once: Safeguarding by Progressive Self-Reflectioncited by 11
  14. 2025Jailbreak Attack Initializations as Extractors of Compliance Directionscited by 10
  15. 2025Amplified Vulnerabilities: Structured Jailbreak Attacks on LLM-based Multi-Agent Debatecited by 10
  16. 2025Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMscited by 9
  17. 2024Cross-Task Defense: Instruction-Tuning LLMs for Content Safetycited by 8
  18. 2025Bridging the Safety Gap: A Guardrail Pipeline for Trustworthy LLM Inferencescited by 8
  19. 2025Dagger Behind Smile: Fool LLMs with a Happy Ending Storycited by 6
  20. 2025When Reject Turns into Accept: Quantifying the Vulnerability of LLM-Based Scientific Reviewers to Indirect Prompt Injectioncited by 6
  21. 2024DistillSeq: A Framework for Safety Alignment Testing in Large Language Models using Knowledge Distillationcited by 5
  22. 2025KDA: A Knowledge-Distilled Attacker for Generating Diverse Prompts to Jailbreak LLMscited by 5
  23. 2024MeTMaP: Metamorphic Testing for Detecting False Vector Matching Problems in LLM Augmented Generationcited by 4
  24. 2024TorchOpera: A Compound AI System for LLM Safetycited by 4
  25. 2024SQL Injection Jailbreak: A Structural Disaster of Large Language Modelscited by 4
  26. 2025When Grammar Guides the Attack: Uncovering Control-Plane Vulnerabilities in LLMs with Structured Outputcited by 4
  27. 2025STShield: Single-Token Sentinel for Real-Time Jailbreak Detection in Large Language Modelscited by 3
  28. 2025VLMInferSlow: Evaluating the Efficiency Robustness of Large Vision-Language Models as a Servicecited by 3
  29. 2025Advancing Jailbreak Strategies: A Hybrid Approach to Exploiting LLM Vulnerabilities and Bypassing Modern Defensescited by 2
  30. 2025Jailbreaking Large Language Models Through Content Concretizationcited by 2
  31. 2025Jailbreaking LLMs via Semantically Relevant Nested Scenarios with Targeted Toxic Knowledgecited by 2
  32. 2026NeST: Neuron Selective Tuning for LLM Safetycited by 2
  33. 2026Patcher: Post-Hoc Patching of Backdoored Large Language Modelscited by 2
  34. 2026NeuronGuard: Robust LLM Safety Alignment via Ablation-Aware Safety Signal Redistributioncited by 2
  35. 2025Turning Logic Against Itself : Probing Model Defenses Through Contrastive Questionscited by 1
  36. 2025Be a Multitude to Itself: A Prompt Evolution Framework for Red Teamingcited by 1
  37. 2025Geometry-Guided Adversarial Prompt Detection via Curvature and Local Intrinsic Dimensioncited by 1
  38. 2025SoK: A Comprehensive Security Analysis of Jailbreak Resilience in GPT and DeepSeek Modelscited by 1

50 most cited of 68 All 68 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025PUZZLED: Jailbreaking LLMs through Word-Based Puzzles
  2. 2025DETAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification
  3. 2024Can LLMs Deeply Detect Complex Malicious Queries? A Framework for Jailbreaking via Obfuscating Intent
  4. 2026Re-Triggering Safeguards within LLMs for Jailbreak Detection
  5. 2024Intention Analysis Makes LLMs A Good Jailbreak Defender
  6. 2023Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks
  7. 2026Did the LLM Answer the Question? A Semantic Reconstruction Game for Jailbreak Evaluation
  8. 2024HSF: Defending against Jailbreak Attacks with Hidden State Filtering