mlsanityData to 7 Oct 2026

Jailbreak Attack Initializations as Extractors of Compliance Directions

arXiv 2502.09755 ↗First posted 13 Feb 2025cs.CR cs.LGcited by 10

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Safety-aligned LLMs respond to prompts with either compliance or refusal, each corresponding to distinct directions in the model's activation space. Recent works show that initializing attacks via self-transfer from other prompts significantly enhances their performance. However, the underlying mechanisms of these initializations remain unclear, and attacks utilize arbitrary or hand-picked initializations. This work presents that each gradient-based jailbreak attack and subsequent initialization gradually converge to a single compliance direction that suppresses refusal, thereby enabling an efficient transition from refusal to compliance. Based on this insight, we propose CRI, an initialization framework that aims to project unseen prompts further along compliance directions. We demonstrate our approach on multiple attacks, models, and datasets, achieving an increased attack success rate (ASR) and reduced computational overhead, highlighting the fragility of safety-aligned LLMs. A reference implementation is available at: https://amit1221levi.github.io/CRI-Jailbreak-Init-LLMs-evaluation.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 45 of its 87 references that are in the map. First posted on arXiv 13 Feb 2025.

Builds on

45 of 87 references.

Closest ancestors · 22

Same direction, newest first

  1. 2024JailPO: A Novel Black-box Jailbreak Framework via Preference Optimization against Aligned LLMs20 Dec
  2. 2024AdvPrefix: An Objective for Nuanced LLM Jailbreaks13 Dec
  3. 2024JailbreakLens: Interpreting Jailbreak Mechanism in the Lens of Representation and Circuit17 Nov
  4. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs3 Oct
  5. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States9 Jun
  6. 2024Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained Optimization15 May
  7. 2024Boosting Jailbreak Attack with Momentum2 May
  8. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
  9. 2024Distract Large Language Models for Automatic Jailbreak Attack13 Mar
  10. 2024Accelerating Greedy Coordinate Gradient and General Prompt Optimization via Probe Sampling2 Mar
  11. 2024Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes1 Mar
  12. 2024Fast Adversarial Attacks on Language Models In One GPU Minute23 Feb
10 more
  1. 2024GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis21 Feb
  2. 2024Defending Jailbreak Prompts via In-Context Adversarial Game20 Feb
  3. 2024Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues14 Feb
  4. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space14 Feb
  5. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability13 Feb
  6. 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective12 Jan
  7. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  8. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  10. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  5. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  6. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  7. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  8. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  9. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  10. 2023The Falcon Series of Open Language Modelscited by 526
  11. 2023Poisoning Language Models During Instruction Tuningcited by 254
  12. 2023Large Language Model Alignment: A Surveycited by 241
11 more
  1. 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judgecited by 202
  2. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  3. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  4. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injectioncited by 143
  5. 2024LLM Agents can Autonomously Hack Websitescited by 108
  6. 2024AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attackscited by 94
  7. 2024Robust LLM safeguarding via refusal feature adversarial trainingcited by 60
  8. 2024Refusal in LLMs is an Affine Functioncited by 26
  9. 2024Evaluating Large Language Models' Capability to Launch Fully Automated Spear Phishing Campaigns: Validated on Human Subjectscited by 17
  10. 2024Lateral Phishing With Large Language Models: A Large Organization Comparative Studycited by 12
  11. 2024Adversarial Attacks on Large Language Models Using Regularized Relaxationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 5.

10 papers.

  1. 2025Silenced Biases: The Dark Side LLMs Learned to Refusecited by 10
  2. 2025Activation Steering for Masked Diffusion Language Modelscited by 7
  3. 2025Silent Tokens, Loud Effects: Padding in LLMscited by 4
  4. 2025You Had One Job: Per-Task Quantization Using LLMs' Hidden Representationscited by 3
  5. 2026Step-Wise Refusal Dynamics in Autoregressive and Diffusion Language Modelscited by 1
  6. 2026Dynamic Adversarial Fine-Tuning Reorganizes Refusal Geometrycited by 1
  7. 2026Linguistic Firewall: Geometry as Defense in Multi-Agent Systems Routingcited by 1
  8. 2025SafeConstellations: Mitigating Over-Refusals in LLMs Through Task-Aware Representation Steeringcited by 0
  9. 2026Unsupervised Features Mining via Activation Geometrycited by 0
  10. 2026Generalization Is Stability, Not Accuracy: Multi-Axis Evaluation of LLMscited by 0