mlsanityData to 7 Oct 2026

Jigsaw Puzzles: Splitting Harmful Questions to Jailbreak Large Language Models

arXiv 2410.11459 ↗First posted 15 Oct 2024cs.CLcited by 15

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Large language models (LLMs) have exhibited outstanding performance in engaging with humans and addressing complex questions by leveraging their vast implicit knowledge and robust reasoning capabilities. However, such models are vulnerable to jailbreak attacks, leading to the generation of harmful responses. Despite recent research on single-turn jailbreak strategies to facilitate the development of defence mechanisms, the challenge of revealing vulnerabilities under multi-turn setting remains relatively under-explored. In this work, we propose Jigsaw Puzzles (JSP), a straightforward yet effective multi-turn jailbreak strategy against the advanced LLMs. JSP splits questions into harmless fractions as the input of each turn, and requests LLMs to reconstruct and respond to questions under multi-turn interaction. Our experimental results demonstrate that the proposed JSP jailbreak bypasses original safeguards against explicitly harmful content, achieving an average attack success rate of 93.76% on 189 harmful queries across 5 advanced LLMs (Gemini-1.5-Pro, Llama-3.1-70B, GPT-4, GPT-4o, GPT-4o-mini). Moreover, JSP achieves a state-of-the-art attack success rate of 92% on GPT-4 on the harmful query benchmark, and exhibits strong resistant to defence strategies. Warning: this paper contains offensive examples.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 36 of its 39 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

36 of 39 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  2. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking16 Nov

Other references · 34

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  6. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  7. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  8. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  9. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
  10. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  11. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  12. 2023Low-Resource Languages Jailbreak GPT-4cited by 290
22 more
  1. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  2. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  3. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
  4. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  5. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 189
  6. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  7. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 176
  8. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 160
  9. 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contextscited by 92
  10. 2024DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakerscited by 91
  11. 2023Safety Assessment of Chinese Large Language Modelscited by 82
  12. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Modelcited by 82
  13. 2023CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibilitycited by 80
  14. 2024Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLMcited by 73
  15. 2023Self-Guard: Empower the LLM to Safeguard Itselfcited by 52
  16. 2024PARDEN, Can You Repeat That? Defending against Jailbreaks via Repetitioncited by 29
  17. 2023Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBenchcited by 28
  18. 2024Sandwich attack: Multi-language Mixture Adaptive Attack on LLMscited by 20
  19. 2023Exploiting Large Language Models (LLMs) through Deception Techniques and Persuasion Principlescited by 17
  20. 2024Emerging Vulnerabilities in Frontier Models: Multi-Turn Jailbreak Attackscited by 16
  21. 2024Self-Evaluation as a Defense Against Adversarial Attacks on LLMscited by 13
  22. 2024Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspectivecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 8.

15 papers.

Sort:

By area:

  1. 2025Multilingual and Multi-Accent Jailbreaking of Audio LLMscited by 25
  2. 2025Jailbreak-AudioBench: In-Depth Evaluation and Analysis of Jailbreak Threats for Large Audio Language Modelscited by 17
  3. 2026SEMA: Simple yet Effective Learning for Multi-Turn Jailbreak Attackscited by 10
  4. 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLMcited by 8
  5. 2026MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safetycited by 7
  6. 2025ASTRA: Autonomous Spatial-Temporal Red-teaming for AI Software Assistantscited by 6
  7. 2026Visual Exclusivity Attacks: Automatic Multimodal Red Teaming via Agentic Planningcited by 4
  8. 2025HarmNet: A Framework for Adaptive Multi-Turn Jailbreak Attacks on Large Language Modelscited by 3
  9. 2026The Echo Chamber Multi-Turn LLM Jailbreakcited by 3
  10. 2025Attention-Aware GNN-based Input Defense against Multi-Turn LLM Jailbreakcited by 2
  11. 2026Not All Turns Matter: Credit Assignment for Multi-Turn Jailbreakingcited by 1
  12. 2026GFlowRL: Scaling Distribution-Matching RL to Large Language Modelscited by 1
3 more
  1. 2025Pluralistic Behavior Suite: Stress-Testing Multi-Turn Adherence to Custom Behavioral Policiescited by 0
  2. 2026GRM: Utility-Aware Jailbreak Attacks on Audio LLMs via Gradient-Ratio Maskingcited by 0
  3. 2026Deny Without Disabling: Authorization-Paired Evaluation and Control for Multi-Agent Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025PUZZLED: Jailbreaking LLMs through Word-Based Puzzles
  2. 2024Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models
  3. 2024Multi-round jailbreak attack on large language models
  4. 2024MRJ-Agent: An Effective Jailbreak Agent for Multi-Round Dialogue
  5. 2025Attack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMs
  6. 2025Jailbreaking in the Haystack
  7. 2025Reasoned Safety Alignment: Ensuring Jailbreak Defense via Answer-Then-Check
  8. 2025Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models