Jailbreak Transferability Emerges from Shared Representations
On the map: LLM safety and security → Jailbreaking
Abstract (arXiv)
Jailbreak transferability is the surprising phenomenon when an adversarial attack compromising one model also elicits harmful responses from other models. Despite widespread demonstrations, there is little consensus on why transfer is possible: is it a quirk of safety training, an artifact of model families, or a more fundamental property of representation learning? We present evidence that transferability emerges from shared representations rather than incidental flaws. Across 20 open-weight models and 33 jailbreak attacks, we find two factors that systematically shape transfer: (1) representational similarity under benign prompts, and (2) the strength of the jailbreak on the source model. To move beyond correlation, we show that deliberately increasing similarity through benign only distillation causally increases transfer. Our qualitative analyses reveal systematic transferability patterns across different types of jailbreaks. For example, persona-style jailbreaks transfer far more often than cipher-based prompts, consistent with the idea that natural-language attacks exploit models' shared representation space, whereas cipher-based attacks rely on idiosyncratic quirks that do not generalize. Together, these results reframe jailbreak transfer as a consequence of representation alignment rather than a fragile byproduct of safety training.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 33 of its 51 references that are in the map. First posted on arXiv 15 Jun 2025.
Builds on
33 of 51 references.
Closest ancestors · 15
Same direction, newest first
- 2025The Jailbreak Tax: How Useful are Your Jailbreak Outputs?14 Apr
- 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming31 Jan
- 2024Boosting Jailbreak Transferability for Large Language Models21 Oct
- 2024Impact of Non-Standard Unicode Characters on Security and Comprehension in Large Language Models23 May
- 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
- 2024Distract Large Language Models for Automatic Jailbreak Attack13 Mar
- 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective12 Jan
- 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
- 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models23 Oct
- 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
- 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
- 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models7 Aug
3 more
Other references · 18
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
- 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
- 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
- 2023Fundamental Limitations of Alignment in Large Language Modelscited by 152
- 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 138
6 more
- 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 114
- 2025Harnessing the Universal Geometry of Embeddingscited by 54
- 2025Shared Global and Local Geometry of Language Model Embeddingscited by 34
- 2024Failures to Find Transferable Image Jailbreaks Between Vision-Language Modelscited by 27
- 2025Eliciting Language Model Behaviors with Investigator Agentscited by 21
- 2024Model Mimic Attack: Knowledge Distillation for Provably Transferable Adversarial Examplescited by 2
Built on by
Citing papers per year: 2025: 2; 2026: 3.
5 papers.
- 2025Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractorscited by 1
- 2025One Leak Away: How Pretrained Model Exposure Amplifies Jailbreak Risks in Finetuned LLMscited by 1
- 2026Steerability of Instrumental-Convergence Tendencies in LLMscited by 1
- 2026Furina: Fragmented Uncertainty-Driven Refusal Instability Attackcited by 1
- 2026Jailbreak Context Lingers: Divergent Safety Routing and Its Cross-Task Predictability in Tool Agentscited by 0
- 2026Jailbreak Context Lingers: Divergent Safety Routing and Its Cross-Task Predictability in Tool Agents28 Sep 2026
- 2026Furina: Fragmented Uncertainty-Driven Refusal Instability Attack24 May 2026
- 2026Steerability of Instrumental-Convergence Tendencies in LLMs4 Jan 2026
- 2025One Leak Away: How Pretrained Model Exposure Amplifies Jailbreak Risks in Finetuned LLMs14 Dec 2025
- 2025Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractors1 Oct 2025