mlsanityData to 7 Oct 2026

TwinBreak: Jailbreaking LLM Security Alignments based on Twin Prompts

arXiv 2506.07596 ↗First posted 9 Jun 2025cs.LGcited by 12

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Machine learning is advancing rapidly, with applications bringing notable benefits, such as improvements in translation and code generation. Models like ChatGPT, powered by Large Language Models (LLMs), are increasingly integrated into daily life. However, alongside these benefits, LLMs also introduce social risks. Malicious users can exploit LLMs by submitting harmful prompts, such as requesting instructions for illegal activities. To mitigate this, models often include a security mechanism that automatically rejects such harmful prompts. However, they can be bypassed through LLM jailbreaks. Current jailbreaks often require significant manual effort, high computational costs, or result in excessive model modifications that may degrade regular utility. We introduce TwinBreak, an innovative safety alignment removal method. Building on the idea that the safety mechanism operates like an embedded backdoor, TwinBreak identifies and prunes parameters responsible for this functionality. By focusing on the most relevant model layers, TwinBreak performs fine-grained analysis of parameters essential to model utility and safety. TwinBreak is the first method to analyze intermediate outputs from prompts with high structural and content similarity to isolate safety parameters. We present the TwinPrompt dataset containing 100 such twin prompts. Experiments confirm TwinBreak's effectiveness, achieving 89% to 98% success rates with minimal computational requirements across 16 LLMs from five vendors.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 41 of its 60 references that are in the map. First posted on arXiv 9 Jun 2025.

Builds on

41 of 60 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
  2. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models28 Mar
  3. 2024A StrongREJECT for Empty Jailbreaks15 Feb
  4. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  6. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  7. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  8. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  10. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots16 Jul

Other references · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2024The Llama 3 Herd of Modelscited by 14,192
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  10. 2023Mistral 7Bcited by 3,065
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  12. 2024Qwen2 Technical Reportcited by 2,076
19 more
  1. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  2. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  3. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  4. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  5. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  6. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  7. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  8. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  9. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  10. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  11. 2020Backdoor Learning: A Surveycited by 412
  12. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  13. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  14. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  15. 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 229
  16. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  17. 2024Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neuronscited by 47
  18. 2024Automated Red Teaming with GOAT: the Generative Offensive Agent Testercited by 41
  19. 2024NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuningcited by 37

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 10.

12 papers.

  1. 2026MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attackscited by 18
  2. 2026GRP-Obliteration: Unaligning LLMs With a Single Unlabeled Promptcited by 6
  3. 2025SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defensescited by 4
  4. 2025QuadSentinel: Sequent Safety for Machine-Checkable Control in Multi-agent Systemscited by 4
  5. 2026NeST: Neuron Selective Tuning for LLM Safetycited by 2
  6. 2026DynaTrust: Defending Multi-Agent Systems Against Sleeper Agents via Dynamic Trust Graphscited by 2
  7. 2026A Causal Perspective for Enhancing Jailbreak Attack and Defensecited by 0
  8. 2026The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoringcited by 0
  9. 2026RoguePrompt: Dual-Layer Encoding for Self-Reconstruction to Circumvent LLM Moderationcited by 0
  10. 2026Decided Upstream, Written Late: Locating and Pricing the Cross-Lingual Refusal Circuit of a Multilingual MoEcited by 0
  11. 2026Localizing Safety Alignment: MLP Layers and Mid-Network Blocks Encode Refusal Behavior in Large Language Modelscited by 0
  12. 2026Fool's Gold: Defensive Deception Against Safety-Removal Attacks on Open-Weight Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SecureBreak -- A dataset towards safe and secure models
  2. 2025Logic Jailbreak: Efficiently Unlocking LLM Safety Restrictions Through Formal Logical Expression
  3. 2025Jailbreaking Attacks vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race?
  4. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States
  5. 2025Align to Misalign: Automatic LLM Jailbreak with Meta-Optimized LLM Judges
  6. 2026MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety
  7. 2025JULI: Jailbreak Large Language Models by Self-Introspection
  8. 2024Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attacks