Toward Understanding the Transferability of Adversarial Suffixes in Large Language Models
On the map: LLM safety and security → Jailbreaking
Abstract (arXiv)
Discrete optimization-based jailbreaking attacks on large language models aim to generate short, nonsensical suffixes that, when appended onto input prompts, elicit disallowed content. Notably, these suffixes are often transferable -- succeeding on prompts and models for which they were never optimized. And yet, despite the fact that transferability is surprising and empirically well-established, the field lacks a rigorous analysis of when and why transfer occurs. To fill this gap, we identify three statistical properties that strongly correlate with transfer success across numerous experimental settings: (1) how much a prompt without a suffix activates a model's internal refusal direction, (2) how strongly a suffix induces a push away from this direction, and (3) how large these shifts are in directions orthogonal to refusal. On the other hand, we find that prompt semantic similarity only weakly correlates with transfer success. These findings lead to a more fine-grained understanding of transferability, which we use in interventional experiments to showcase how our statistical analysis can translate into practical improvements in attack success.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 29 of its 44 references that are in the map. First posted on arXiv 24 Oct 2025.
Builds on
29 of 44 references.
Closest ancestors · 11
Same direction, newest first
- 2024Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis16 Jun
- 2024Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models13 Jun
- 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
- 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models28 Mar
- 2024Attacking Large Language Models with Projected Gradient Descent14 Feb
- 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models23 Oct
- 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
- 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
- 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
Other references · 18
- 2023GPT-4 Technical Reportcited by 18,132
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2013Intriguing properties of neural networkscited by 5,114
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
- 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
- 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
- 2016Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samplescited by 439
- 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
- 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
6 more
- 2018Audio Adversarial Examples: Targeted Attacks on Speech-to-Textcited by 268
- 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
- 2017The Space of Transferable Adversarial Examplescited by 156
- 2024What Makes and Breaks Safety Fine-tuning? A Mechanistic Studycited by 54
- 2018Targeted Adversarial Examples for Black Box Audio Systemscited by 41
- 2019Universal Adversarial Perturbations for Speech Recognition Systemscited by 41
Built on by
Not cited yet.